Cyber Panel

Update 878787: Total failure of a hosting provider

I never thought that one of the hosting providers I use for my VPS’s would become the successful target of a hacker attack. But that’s what happened: On 04.01.2025, this website was suddenly gone. The entire VPS on which CyberPanel was running was no longer accessible. After a while, it turned out that the attacker had attacked the Proxmox server and destroyed the entire cluster system with all virtual hard disks and all VPSs, including the snapshot backups. Initially, it was said that attempts were being made to restore the hard disks. However, the incident had already been reported to the police and the Federal Office for Information Security (BSI), the Federal Network Agency (BNetzA) and the state data protection authority, and they naturally forbade any attempt to recover the data, as this would cover all traces. The criminal investigation department then confiscated the physical hard disks and handed them over to a forensics department. They will remain there for the next two years, and the probability of being able to recover data after that is said to be very low.

Apart from the fact that their Proxmox system was apparently handled quite carelessly, as they allowed remote access to the core in the first place and, it seems, the firewall was also inactive at the time. But the real kicker was that they had zero external backups or offline backups. Zero! All data gone, completely. I didn’t get anything back for the three VPSs I was running there. I’ve been working on this website, for example, for years. A lot of work and accumulated knowledge went into it. If I had blindly trusted the provider, everything would have been gone, forever.

At the time, I had migrated my website from an expensive provider to a cheaper one, which has now also gone down. Even though this turned out to be a mistake in hindsight, I learned a lot in the process, which has now saved me. I knew how to move CyberPanel and my website and how to back up my data externally. This knowledge and the experiences from the acid test, which I will discuss later, have now also saved me. I had a basic manual backup of the website directory from the beginning of November 2024 and an automatically created UpdraftPlus Cloud Backup from December 29, 2020. And luckily, I had been able to quickly make an offline website backup of the site in my browser that I had been working on for the last few days, so I didn’t have to reinvent everything, but just had to copy it back and upload the few images again.

You see, if you rely on others, there is a greater than zero chance that you will regret it. Don’t rely on local backups, distribute the backups to other independent systems. Then you can always sleep soundly. After this incident, I have revised these backup and recovery instructions again. I hope it will save one or two people a lot of trouble.

Picture: Annoucment (Translation) – Late realization

The acid test – A cyberattack on my CyberPanel server

After receiving an abuse report from my hosting provider on 28.10.2024, a few exciting days began. A security researcher discovered an RCE vulnerability (Remote Code Execution) in the software of my self-hosted web hosting panel CyberPanel (GitHub). Unfortunately, he also published an exploit in his blog post dated 27.10.2024. It didn’t take long before thousands of web hosting servers and even more websites were infected. Crypto-mining malware, ransomware that encrypted files as .psaux, .encryp, .locked, etc., or took over the server or rendered it unusable.

Image: Notification message from a server affected by ransomware.
The ransom demands were also paid in Bitcoin, by the way.

The need was great, and the precautions for my website proved to be rather inadequate. Who would have thought that a small private website would become the target of a major attack? Well, anyway, I was lucky. I had various backups and knew how to restore them because I had migrated from one hosting provider to another a few weeks earlier. Still, it took a while and cost a lot of nerves. I learned a lot in the process and realized that every Virtual Private Server must be secured to the maximum. In this article, I have loosely compiled a few points. Maybe they will help someone.

1. Set up and harden the VPS

Note: This tutorial uses an Ubuntu 22.04 LTS server. You can find a guide to setting up an Ubuntu VPS here, for example: 7.1 – General – Setting up a Virtual Private Server (VPS). The minimum requirement is 2 GB RAM.

Root Login:

ssh root@yourIPaddress

1.1 Basics

Update / Upgrade

apt update && apt upgrade -y

-> You may need to press Enter.

Note: The ufw (uncomplicated firewall) is not installed here because a firewall (firewalld) will come later with CyberPanel.

fail2ban

apt install fail2ban -y 
systemctl enable fail2ban
systemctl start fail2ban
systemctl status fail2ban

-> Confirm twice with Enter.

Note: Swapfile is not installed here because it will be set up automatically later with CyberPanel.

1.2 Create a dedicated user

For greater security, we create a new user who will be used for future logins. Although this user also has „root“ privileges, they must always identify themselves with their password. This provides more security, better traceability, and protection against human errors.

adduser cypherpunk

-> Set a password; the rest of the data input is optional.

usermod -aG sudo cypherpunk 

-> Assign root privileges to the user.

Important: Edit the OpenSSH server configuration file.

sudo nano /etc/ssh/sshd_config  

Add the following line:

AllowUsers cypherpunk

-> CTRL+X -> Y -> Enter

Note: If you still want to access the VPS with the „root“ user, you must also add the line AllowUsers root to the configuration file if it is not already present. Otherwise, when attempting to log in as the „root“ user, you will see the error message „Permission denied, please try again.“ After five failed attempts, access will be completely blocked for 10 minutes as Fail2ban blocks all connections.

Now shut down the VPS, optionally take a snapshot, and restart everything from scratch.

sudo systemctl poweroff

-> To create a backup of your Virtual Private Server as a snapshot, you will surely find a corresponding function on the website of your VPS hosting provider. This creates a complete image of your disk or VPS. You can restore this at any time to revert the server to a previous state. Then restart the server via the website and log in with the new user from now on.

ssh cypherpunk@yourIPaddress

2. Install and Configure Cyber Panel

2.1 Installing Cyber Panel

Note: To follow this tutorial, the hostname of the server should not match your domain name. So, do not use yourdomain.xyz. Instead, name the host something like „CyberPanel“.

sudo su - -c "sh <(curl https://cyberpanel.net/install.sh || wget -O - https://cyberpanel.net/install.sh)"

-> During the installation: 1+Enter, 1+Enter, 6x Enter

Attention: At the end of the installation, the CyberPanel access credentials will be displayed:

   ...
   CyberPanel Successfully Installed
   ...
   Visit: https://yourIPaddress:8090
   Panel username: admin
   Panel password: bAJWFpasswordWEjV6
   ...

-> After saving the data, confirm the message Would you like to restart your server now? [y/N]: with ENTER.
-> sudo systemctl poweroff and create a snapshot.

2.2 Setting up Cyber Panel

After logging into CyberPanel, you will see the „Set-up Wizard“ at the top. This helps you set up the first website. Ideally, the A and AAAA records already point to the IP address of the CyberPanel server. Log into CyberPanel and follow these steps.

1. If available, use the „Set-up Wizard“ displayed at the top. It simplifies setting up the first website.
2. If it is no longer displayed, you can manually set up the website under MAIN > Websites > Create Website if not already done.
3. Create an FTP account on the new server under MAIN > FTP> Create FTP Account . The user name “yourusername” is automatically changed to “admin_yourusername”. The IP address is that of your VPS, the port is 21, and you can use FileZilla as the FTP client.
4. If you are setting up CyberPanel anew to migrate an existing version, after restarting CyberPanel, you can preconfigure the PHP version under MAIN > Websites > List Websites > Manage > Change PHP. You can find the version in the „Website Status Report“, see point 3.1 / guideline point 5.

3. Data Backup and Migration

Before we can restore or migrate data, we first need to back it up.

3.1 Data Backup

To ensure that your website can be quickly restored in the event of a problem, I recommend at least three methods of data backup.

3.1.1 Snapshots of the Entire VPS

Regularly create snapshots of the entire VPS. These can be done manually, especially before updates or significant changes, and can also be scheduled. The exact procedure varies depending on the hosting provider and is usually well documented. It generally works very well and reliably. However, it is of no use if you want to migrate to another VPS or if your VPS or snapshot is deleted because your hosting provider was exposed to a hacker attack and has no backups, or if the hard drive for data recovery is in the forensic department of the criminal police.

Regular snapshots of the entire VPS are an excellent way to back up the state of your server. These can be created both manually, e.g., before updates or major changes, and automatically on a schedule. The exact procedure depends on the hosting provider and is generally well documented. Snapshots usually work reliably and are a quick way to restore the VPS to its previous state.

Important Note: However, snapshots are not very useful if you want to migrate to another VPS or if the VPS or snapshot is deleted – for example, if your hosting provider was the victim of a hacker attack and has no backups, or if the hard drive for data recovery is being held by the police in the forensic department as part of a criminal investigation. This is what happened to me, at least.

Therefore, I recommend also using external backup methods to create a more secure foundation for your data.

3.1.2 Automatic Backups to External Cloud Drives

Use plugin software such as UpdraftPlus to create scheduled backups of your WordPress website and upload them to external cloud storage services like Google Drive, Dropbox, or OneDrive. Detailed instructions for setting this up can be found from the respective plugin providers. Automatic backups secure all WordPress-related data, such as the database, themes, and plugins. This method is very reliable and offers a convenient way to protect essential data.

Important: However, automatic backup methods do not cover all data. Manual entries in the directory, made via FTP uploads or through the CyberPanel menu, are usually not included in these backups. These files remain unsecured, which could lead to problems if they are lost or needed during a migration.

3.1.3 Manual Backup of Data, Database, and Website Status Report

Manual backups are particularly useful if you want to create backups independently of automated tools. It is ideal for migrating the website to a different hosting provider or to a new VPS if needed. Here’s a short guide:

1. Create a Folder
   Create a folder on your local computer with the name in the format YYMMDD (e.g., 250111 for January 11, 2025).
   
2. Backup Website Files

-> Navigate to the CyberPanel web server: MAIN > Websites > List Websites > yourdomain.xyz > File Manager.
-> Compress the public_html folder into a .tar.gz file.
-> Download the public_html.tar.gz file and save it in the folder you previously created.

3. Export MySQL Database

-> Open phpMyAdmin in CyberPanel under MAIN > Database > phpMyAdmin.
-> Select your website’s database from the left column (e.g., your_databasename).
-> Export the database with the settings „Quick“ and „SQL“.
-> Save the your_databasename.sql file in the backup folder.

4. Save Website Status Report

-> Open WordPress and navigate to Tools > Site Health > Report.
-> Copy the „Site Health Report“ to the clipboard.
-> Create a text file in the backup folder and paste the report there. Save the file to keep important information such as the PHP version, plugins used, and other details.

Recommendation: To also secure these manually uploaded files, the next method is recommended: manual backups. This method ensures that all content – including custom files that are not automatically backed up – is also stored externally.

Important Note: Clean the Database Before Backup

Before exporting the database, you should clean it to avoid potential complications during later import. Especially for older websites that have gone through many updates and upgrades, there can be accumulated data junk, countless revisions, and broken links. These can cause conflicts during the import. Steps for cleaning:

1. Use Plugins for Cleaning:

-> Use a plugin like WP-Sweep, WP Optimize, or Advanced Database Cleaner to remove unnecessary entries from the database.
-> This not only saves storage space but also reduces the likelihood of conflicts.

2. Tools in phpMyAdmin:

-> An additional cleaning tool is available in phpMyAdmin. To do this, select the database, scroll down, “Select all” and then select “Utilities > Repair table” from the drop-down menu. Then repeat the same with “Utilities > Optimize table”. However, this alone is often not enough, so it is recommended to combine it with one of the plugins mentioned above.

Tip: Backup data externally in WordPress format in between. If you want to back up the most recent formatting and content of individual pages, you can do that directly in the WordPress Block Editor:

1. Open the desired page in the Block Editor.
2. Switch to the Code Editor in the top-right corner (Editor > Code Editor).
3. Select all the source code of the page and copy it to the clipboard.
4. Save the code as a simple text file on your computer.

This method is particularly useful if you want to document smaller but important changes or ensure that no work is lost.

3.2 Data Recovery / Data Migration

3.2.1 Transfer Data

Restoring data from a snapshot with a VPS hosting provider is relatively simple and well-documented by most providers. The restoration using UpdraftPlus is also well described. However, manually migrating the backup and database requires a few additional steps that you need to be familiar with. Here is a guide on how to restore data from the backup folder (backup method in Chapter 3.1.3) to a newly set up server:

1. Upload and Extract Files

Using FileZilla, upload the public_html.tar.gz file to the /home/yourdomain.xyz directory on the new server. Log in to CyberPanel and navigate to the /home/yourdomain.xyz directory. Click on the „Extract“ button to unpack the file into the public_html directory.

2. Upload Database

Also, upload the database file your_databasename.sql using FileZilla to the /home/yourdomain.xyz/public_html directory.

3.2.2 Migrate Data

1. Create an empty database with Database > Create Database:

Enter -> databasename, username and password
databasename: your_databasename
username: your_username
password: …

2. Log as „root“ user into the server using the Command Line Interface (CLI) and import the database:
mysql -u your_username -p your_databasename < /home/yourdomain.xyz/public_html/your_databasename.sql
Note: The password is for the new empty database that was created earlier, not the old database.

3.2.3 Configure Database Connection

1. In CyberPanel, open the file public_html/wp-config.php in the File Manager of the site.
2. Set the new values for DatabaseName, UserName, and Password, and ensure the host is set to localhost.

Check the PHP version used from the „Website Status Report“ in the backup and, if necessary, configure it under MAIN > Websites > New Websites > Manage > Change PHP.

3.2.4 Final Steps

1. Don’t forget: If you have a new server with a new IP, update the IP forwarding as well. At your domain provider, set the A and AAAA records to the new VPS IP with CyberPanel. Then check, for example, with: nslookup.io.
2. Restart the VPS once with systemctl reboot.
3. To test, you can create a text file named neu.txt in ./public_html/. You should be able to access the file in your browser with yourdomain.xyz/neu.txt.
4. If you still get TLS/SSL error messages, issue a certificate in CyberPanel under MAIN > SSL > Manage SSL > Issue SSL.

4. Options for Advanced Hardening

The following methods to further enhance security are relevant and effective. However, errors often occur, and you might lock yourself out. First, create a snapshot of your VPS so you can always revert back. Then proceed very slowly, step by step, testing the new access before disabling anything. Also, remember that CyberPanel includes its own firewall, which you can configure directly in CyberPanel. For example, you can allow or close individual ports for SSH access there.

4.1 Use SSH Key

Using SSH key pairs might seem complicated and confusing at first, but once you understand it, it’s quite straightforward. Let ChatGPT explain:

An SSH key pair is a type of security tool used for securely logging into a computer or server. Think of it as a lock and a matching key that work together to open a door—only here, it’s about digital security.

Here are the two parts of the SSH key pair:

1. Private Key: This is like your personal, secret key. You should never share it with anyone. It always stays secure on your computer.

2. Public Key: This is like the lock you install on the server. It can be seen by others, as it doesn’t matter if it is public. The public key is copied to the server you want to log into.

How does it work?

1. When you want to log into a server, the server checks your public key.
2. Your computer (with the private key) "communicates" with the server, and the server ensures that the private key matches the public key.
3. If they match, you are logged in.

This method is secure because only the person with the matching private key can log in. Since the private key remains secret and is never sent, the likelihood of a hacking attack is much lower than with a password.

You can easily generate a key pair on your computer yourself. Open your terminal as if you were logging in, and then generate a key pair with the following command:

ssh-keygen

First, you will be asked where you want to save the key pair. If you simply press ENTER, the suggested path and file name in brackets will be used. Then you’ll be prompted to „Enter passphrase“. Here you can encrypt the key with an additional password. This passphrase is the password to decrypt the private key. When you log into the remote server, the private key must be decrypted with the passphrase. However, you will no longer need the user password to log in. The passphrase is optional but offers additional security. Make sure not to lose the key file or the passphrase. Below is an image showing how to generate an SSH key pair on a Windows 11 computer.

Image: Generating an SSH key pair

Ed25519 is a modern asymmetric encryption algorithm. The private key id_ed25519 and the public key id_ed25519.pub are stored on your computer in the path C:\User\User\.ssh. They are two files you can open and read with a simple text reader like Notepad. Alternatively, you can use the cat id_ed25519.pub command to display the key in the terminal. The know_hosts file is created by your computer to store private keys of other computers. It serves only to verify the authenticity of the other computer.

Quick Note: If you reinstall your server, it generates a new key pair, and you might see the following warning message.

Image: Warning message „man-in-the-middle attack“

In this case, you can stay calm because you now know why the message appears: the key changed with the reinstallation. To resolve this, you just need to remove the key from the know_hosts file. When you log in again, your computer will save the server’s new key in this file. The warning message also indicates where the file is located. In this case: C:\Users\axels\.ssh.

Let’s continue… Now you can read the contents of the public key id_ed25519.pub, copy it to your clipboard, and paste it into the authorized_keys file on your CyberPanel server:

mkdir ~/.ssh
nano ~/.ssh/authorized_keys

-> CTRL+X -> Y -> ENTER.

Now you can test the key by logging out with exit and then logging back in. Either you’ll log in as if by magic, or, if you’ve set a passphrase for the key, you’ll be asked for it to decrypt the private key before being logged in.

Important: You need to save the keys for both the root and cypherpunk users. Log in to both accounts and save the keys.

Note: If you press ENTER at the SSH key login prompt, it will fall back to a password prompt. Try it out!

Note: You can now view the SSH keys you’ve added in CyberPanel under SERVER > Security > Secure SSH > SSH Keys.

4.2 Disable Password Login

If you have secure access using your SSH key, you can disable password login. Before doing so, set up a second device as a backup to access the server if needed. Alternatively, you can securely store the private key id_rsa (on Windows). The public key is not sensitive, but the private key is.

To disable password login:

sudo nano /etc/ssh/sshd_config

Set PasswordAuthentication to no:

PasswordAuthentication no

-> CTRL+X -> Y -> ENTER

Restart the SSH server:

sudo systemctl restart ssh

-> Log back in and skip the SSH key prompt by pressing ENTER. You should no longer see a password prompt. This applies to both users.

4.3 Disable Foreign SSH Keys

In the .ssh directory of the root user, there is a private and public key pair named cyberpanel. I couldn’t find a use for them, so I disabled them.

nano ~/.ssh/cyberpanel
nano ~/.ssh/cyberpanel.pub

-> Add „NO_“ at the beginning of each key to invalidate them.

4.4 Disable Root User Login

The root user is a known target for attacks, so you can disable its login. You have two options: disable it through the CyberPanel firewall under SERVER > Security > Secure SSH > Basic > Permit Root Login, or directly via the console:

sudo nano /etc/ssh/sshd_config

PermitRootLogin no
# AllowUsers root

-> CTRL+X -> Y -> ENTER

Note: With # AllowUsers root commented out, toggling the „Root User“ option in CyberPanel is also disabled. If you want to manage this through CyberPanel, remove the # in front of AllowUsers root. Then only the yes/no of PermitRootLogin applies.

Restart the SSH server:

sudo systemctl restart ssh

-> Try logging in as root. It should no longer work. Only the cypherpunk user should now work with the SSH key.

4.5 Customize Port Access (SSH Port 22 -> 1001)

You can freely choose the new SSH port for access, for example, port number 1001. Since CyberPanel manages the firewall and ports for you, you also need to enable the port there. Go to CyberPanel under SERVER > Security > Firewall and set the rule:

Rule Name = SSHnew / tcp / 0.0.0.0/0 / Port = 1001

-> You should now see the new port rule in the list.

Next, add the port to the sshd_config file:

sudo nano /etc/ssh/sshd_config

Add an additional line for the port:

Port 1001 

-> CTRL+X -> Y -> ENTER

sudo systemctl restart ssh

Log out with exit and log back in using a slightly modified command:

ssh -p 1001 cypherpunk@yourIPaddress

-> Note the -p 1001. The port release for 1001 now applies to all users.

Now disable the default port 22 in the CyberPanel firewall. Go to SERVER > Security > Firewall and delete the entry for port 22.

Next, disable port 22 access in the SSH config:

sudo nano /etc/ssh/sshd_config

Comment out the line for port 22 by adding a #:

# Port 22

CTRL+X -> Y -> ENTER

Restart the service:

sudo systemctl restart ssh

Reboot the server, log out with exit, and log back in to check access, hoping you haven’t locked yourself out. 😉

This adds a significant layer of security to your server. 💪 You’ve left just one small hidden door open. Keep the access and knowledge about it secure.

4.6 Close Unused Ports

Depending on how you use CyberPanel, you may require various ports to remain open. However, if you don’t use certain features like email, you can close all related ports. Other ports are optional. Personally, I only use the basic functions, which include ports: 8090, 80, 443, 21, and 22 or 1001. All other ports are closed or removed from the firewall settings. To do this, go to SERVER > Security > Firewall and delete all unnecessary ports.

4.7 Disable CyberPanel

What isn’t running can’t be exploited. The websites themselves run without an active CyberPanel.

Here are some commands to stop, disable autostart, start, enable autostart, and check the status. You can disable CyberPanel anytime if it’s not needed. Here are some useful commands. „Stop“ only applies to the session; CyberPanel will be active again after a reboot. „Disable“ will permanently disable CyberPanel. „Enable“ will re-enable it. „Status“ shows the current status.

sudo systemctl stop lscpd
sudo systemctl disable lscpd
sudo systemctl start lscpd
sudo systemctl enable lscpd
sudo systemctl status lscpd

4.8 New user name and password for OpenLiteSpeed

The OpenLiteSpeed Panel can be accessed by directly calling the server and port 7080: https://yourIPaddress:7080. To change the user and password, log in with the “root” user and call the script ./admpass.sh.

cd /usr/local/lsws/admin/misc
./admpass.sh

5. Appendix

5.1 Sources and Further Information

Website: https://www.cyberpanel.net
Forum: https://forums.cyberpanel.net or https://community.cyberpanel.net/
Wiki: https://docs.cyberpanel.net
Documentation: https://cyberpanel.net/docs/
GitHub: https://github.com/usmannasir/cyberpanel

5.2 View and Upgrade CyberPanel Version

Check CyberPanel version and commit:

cd /usr/local/CyberCP/
sudo git log -1

You can check for new versions and commits in the GitHub repository: https://github.com/usmannasir/cyberpanel

Upgrade CyberPanel:

sudo su - -c "sh <(curl https://raw.githubusercontent.com/usmannasir/cyberpanel/stable/preUpgrade.sh || wget -O - https://raw.githubusercontent.com/usmannasir/cyberpanel/stable/preUpgrade.sh)"

-> Now restart the entire server so the firewall is reactivated: sudo systemctl reboot

Note: If you encounter the following error during the upgrade:

 TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'

Perform a Pip package update:

sudo pip install packaging==22

The complete error message can be found at pastebin.com.

5.3 Ubuntu System Update

sudo su -
apt update && apt upgrade -y
reboot

5.4 fail2ban Log Files

# Show simple logfiles
sudo cat /var/log/fail2ban.log
# Show sorted logfiles
sudo awk '($(NF-1) == "Ban"){print $NF}' /var/log/fail2ban.log \
  | sort | uniq -c | sort -n

For more information, visit here.

5.5 Check CyberPanel Firewall

You can view the CyberPanel firewall status under SERVER > Security > Firewall. Check it directly and view logs via the command line:

sudo systemctl status firewalld
sudo journalctl -u firewalld

The SSH port settings, root login, and stored SSH keys can be viewed or changed under SERVER > Security > Secure SSH.

Erstellt mit Liebe 🧡 Seit 858407 / 878787

– Lightning ⚡ (er)leben –

Value 4 Value
axelhamburch@ereignishorizont.xyz