On Sunday morning, we got an anomaly alert. It was March 19, 2023. This story is about what happened.

Background

The Blue Core Research website uses WordPress (a free and open-source content management system). WordPress usually uses MySQL as a backend database, and our installation is no different.

While our WordPress doesn’t contain sensitive data, we still protect its database with Core Audit. We do it both to protect our server and prevent a breach from internet attack, and to use our own software.

As you’ll see in this true story, protecting the database also protects the application. WordPress happens to be the application we protected, but this defense is effective for any application.

The Anomaly

Our Core Audit installation has a relatively simple configuration with daily anomaly detection alerts. We get a few false positives every few days, but on Sunday morning of March 19, 2023, we got an alert of a new error:

Access denied for user ''@'' (using password: YES)

It immediately raised a red flag in my mind. That’s because anomalies only alert of something different, like a change in the application activity profile. But a change that causes an access-denied error seems suspicious.

The Investigation

Core Audit anomalies rely on the Core Audit Security Repository. And this particular anomaly is based on the Reduced SQL portion of that repository. The Reduced SQL repository contains every SQL construct executed in the database at a 5-minute resolution.

Once I got the alert, I logged into Core Audit and looked at the relevant forensic view. I quickly found what I was alerted about:

Date:     Sat 2023/03/18
Time:     18:55 - 19:00
Count:    1 per 5 minutes
Rows:     0
Command:  ERROR TEXT
Activity: Access denied for user ''@'' (using password: YES)

That is an internal MySQL error resulting from a failed login. The user and host in this error are between single quotes (‘) and are, therefore, stripped from the Reduced SQL repository. Striping literals is part of how the Reduced SQL repository operates.

My next stop was to look for the failed session that caused it. Switching to the session forensic view, I found this session:

Start:    2023/03/18 18:55:59
End:      2023/03/18 18:55:59
Type:     No Login
Username: username_here
Machine:  localhost

That seems unusual since we don’t have a user in the database called username_here. That is, obviously, an invalid username. My next stop was to look at the web server logs to see who or what triggered this unusual login.

Here is an excerpt from the Apache log:

[18:55:36] GET /.wp-config.php_copy
[18:55:38] GET /.wp-config.php.rar
[18:55:39] GET /.wp-config.php.7z
[18:55:41] GET /.wp-config.php.tmp
[18:55:42] GET /.wp-config.php_tmp
[18:55:44] GET /.wp-config.php.old
[18:55:46] GET /.wp-config.php.0
[18:55:47] GET /.wp-config.php.1
[18:55:49] GET /.wp-config.php.2
[18:55:50] GET /.wp-config.php.zip
[18:55:52] GET /.wp-config.php.gz
[18:55:53] GET /.wp-config.php~
[18:55:55] GET /wp-config.php.templ
[18:55:56] GET /wp-config.php1
[18:55:58] GET /wp-config.php2
[18:55:59] GET /wp-config-sample.php
[18:56:00] GET /wp-config-backup.txt
[18:56:02] GET /wp-config.php.orig
[18:56:03] GET /wp-config.php.original
[18:56:05] GET /wp-license.php?file=..%2F..
[18:56:06] GET /wp-config.save
[18:56:07] GET /wp-config.txt
[18:56:09] GET /wp-config.dist
[18:56:10] GET /.wp-config.php.swo
[18:56:12] GET /wp-config%20copy.php
[18:56:12] GET /wp-config_good
[18:56:14] GET /wp-config-backup
[18:56:15] GET /wp-config-backup.php
[18:56:16] GET /wp-config-backup1.txt
[18:56:18] GET /wp-config-good
[18:56:19] GET /wp-config-sample.php.bak
[18:56:21] GET /wp-config-sample.php~
[18:56:22] GET /wp-config.backup

The culprit is clearly wp-config-sample.php, and looking inside this PHP file, we can find these lines:

define( 'DB_USER', 'username_here' );
define( 'DB_PASSWORD', 'password_here' );
…
require_once ABSPATH . 'wp-settings.php';

We now understand what happened: someone scanned the website for vulnerabilities and called wp-config-sample.php. That PHP script contains an invalid user and password that triggered a failed connection to the database. Core Audit detected this unusual behavior and raised an alert.

WordPress?

While many won’t consider WordPress a good example of application security, we think it’s valuable because of several important factors:

3rd Party & Supply Chain

First and foremost – WordPress is a 100% third-party application. We didn’t code it, have not reviewed the source code, and have almost no knowledge of the code, the data model, or any other aspect of this application.

Being open-source software, WordPress is exposed to both vulnerabilities hackers may discover in the source code and to supply chain attacks.

To make matters worse, WordPress websites almost always use plugins. Much of the power of the WordPress platform is in its extensibility by the tens of thousands of plugins that are out there. Plugin use significantly expand the surface area of 3rd party software and supply chain attacks. Plugins mean a lot of additional unknown code from multiple vendors or developers, different coding standards, enhancements to the data model with more tables, and more.

In other words – WordPress is what you might consider and extreme example of a 3rd party application and protecting it is more challenging than most other applications.

Dynamic SQL

WordPress takes the concept of dynamic SQL to an extreme. Not only do all the SQLs embed literals, but SQLs are often dynamically generated with where clauses created on the fly based on user input.

The challenge is not only dynamic SQL attacks but also the large SQL vocabulary where it’s impossible to predict all the SQL combinations WordPress may generate.

Again, plugins worsen the problem due to an even larger and unknown SQL vocabulary and more potential vulnerabilities in the code that generates those dynamic SQLs.

Popular & Internet facing

WordPress is very popular and is an internet-facing application. Hackers are, therefore, highly incentivized to find vulnerabilities. Exploiting such vulnerabilities can also be easy since the internet is full of WordPress websites with little or no additional security.

Bottom line – we provided effective defense to a large, dynamic, and extensible application like WordPress without relying on known vulnerabilities or the specifics of the application.

Resolution

Once we knew what the attack was, the next challenge was finding a way to protect the application from this and similar attacks.

In WordPress, the file the was attacked (wp-config-sample.php) is part of the WordPress installation package. The WordPress installation process copies that file into wp-config.php and defines some parameters like the database user and password in the copy.

Therefore, wp-config-sample.php is expected to exist in any WordPress installation. And if the WordPress development team has done its job right, there is probably no means of breaching WordPress by calling it.

The attacker was clearly targeting some vulnerability in the code. And even if that vulnerability doesn’t exist in our installation today, as we all know, bugs exist, and zero-day attacks can always occur. Having that file sitting around seems like asking for trouble. That file serves no purpose in WordPress other than as a template for wp-config.php, so shouldn’t we delete it?

That takes us to another WordPress behavior – the WordPress update. When WordPress updates, it overwrites all the files that are part of the WordPress installation package. Therefore, deleting wp-config-sample.php will be undone as soon as WordPress automatically updates.

Another option is to leave the file in place but change file permissions so that Apache can’t access it. That would prevent that PHP script from executing, but may also cause errors when WordPress attempts to update and overwrite it.

The best solution might be to deny access to it using .htaccess:

<Files ~ "wp-config-sample.php">
deny from all
</Files>

Going Further

As previously explained, wp-config-sample.php is used to create wp-config.php during installation. That means that the copy (wp-config.php) could also be vulnerable to the same attack or other similar zero-day attacks.

More troubling is that if a WordPress update fixes a vulnerability in wp-config-sample.php, that fix will not update in the wp-config.php copy since that copy is never modified.

wp-config.php is meant to be included by other WordPress files like index.php. After defining the database connection parameters, the script calls wp-settings.php to set up the WordPress environment, including a database connection. That’s how we got a connection to the database during the attack.

But wp-config.php is never meant to be called directly. To protect it against direct execution attacks like we saw in this attack, we can add a line at the top of wp-config.php to stop the processing on direct execution:

if (get_included_files()[0] == '/path_to_site/wp-config.php') exit();

Final thoughts

Despite the challenges presented by protecting an application like WordPress, Core Audit anomaly analysis provides effective security. It has a reasonably low false positives level and, in this case, has alerted of an attack against the application.

It’s important to note that Core Audit doesn’t have special support for PHP or WordPress. It doesn’t use signatures and this attack could have been a zero-day vulnerability. Also, our configuration wasn’t looking for any particular vulnerability in the database or the application. We only had a general-purpose anomaly detection that alerted us as soon as the application behaved differently. That was enough to identify the attack.

As you’ve seen, monitoring changes in the database activity profile of the application lets us detect many changes in the application behavior. While some changes occur naturally, others indicate a security problem or an attack.