Is it a good idea to use GPTs/LLMs for computer security?

How to use a WordPress hack with ChatGPT help

In today's article we will see how an LLM, ChatGPT, helped me clean up a hacked WordPress.

It seemed like a simple task.

A close friend asked me for help with his WordPress because a strange message appeared at the top of the header of any page that said something like “Order allow,deny Deny from all Order allow,deny Deny from all” .

Apparently, he found in the support forums that it was a problem with a pending update that had the issue of his “Blog” (in quotes because it is more of a service sales page, although it also has a blog and podcast and one day it will have a store… you know… nowadays WordPress can be as versatile as you want, of course… relying on a bunch of third parties and companies… and this is when, if you don't know what you're doing, it goes haywire) .

It seemed like an easy job, make backups before updating, problem fixed and woe. Poor me! I forgot an important motto that is recited in the field of developers: “Developers do things because we think they are easy to do” (and then they are not, but we have already started…)

I find an updated WordPress, yes, but with more than a dozen outdated plugins, obviously the theme, and all the default WordPress themes, which are not few, outdated as well.

Plugins that are activated but not configured, plugins that apparently duplicate the functionality of other configured plugins, and plugins that don't even have any meaning to their existence (more on this later) .

Prerequisites

The first problem, I don't have SSH access, you have to pay separately and seeing that and that half of the WordPress administration pages don't work, it gives a suspicious 403 Forbidden, so we make the corresponding payment to have an SSH connection. And the thing is that cheap things, very often, turn out to be expensive.

One of the premises was not to start updating things without having secured backups, the page could not "fall", since potential customers who might appear, would not give you a second chance.

Despite the error message, the website was operational for its intended purpose (we will ignore the possibility that any data in the database was compromised, but luckily, there were no comments or any potential customer data saved there, because it was going out with the plugin on duty... although you know what).

This complicated the task, since seeing the level of hacking later, I would have preferred to do a clean installation, but it was a guarantee of having to "close" the site until the administrator had all the desired customizations and integrations again.

So nothing, let's do "semi-manual" cleaning, little by little and knowing that we are at constant risk until everything is cleaned and access is protected and any credentials are rotated.

Benign or malignant?

Let's get to work. The first suspicious thing I found was a .htaccess everywhere that said:

<FilesMatch '.(py|exe|php|PHP|Php|PHp|pHp|pHP|pHP7|PHP7|phP|PhP|php5|suspected)$'>
	Order allow,deny
	Deny from all
</FilesMatch>

I asked GPT to explain the file to confirm that it did what I thought and... the explanation was good, the identification that it was harmful... nothing at all... I didn't give it priority to fix this, even though I was clear that it didn't smell right, with so many and so complex plugins installed, let's see if it was useful for anything.

In short, it's a good practice in general! And I fell on all fours! Actually no, I double-checked later, we'll see.

This shouldn't be there.

Before messing with htaccess I saw, clearly with the naked eye, that the server was compromised, there were folders with fairly random names with files with also random names with harmless extensions like .png, .jpg, etc. But clearly this wasn't going there.

Going back to the original error, the “order deny” and all that, I looked through some WordPress page header construction files and came across a wonderful first line.

We are starting to find interesting things...

Plus, the bastard is hiding inside the well-known anti-spam plugin Akismet, how ironic...

Thanks to GPT I can clearly see what the guy is doing, I agree that decoding a base64 is nothing out of the ordinary, but it makes more sense if it's explained well.

And what do you do?

What was that image and what did it do? I asked the GPT for help, I downloaded it, but I didn't dare open it with anything, lest they infect me in the process :P

Luckily he taught me a couple of tricks:

Indeed, all suspicions were confirmed:

A little Git for traceability

So since I didn't have anything "closed" yet, I needed a way to track if the cleanup was going well or if they had immediate ability to restore the crap. So I put all the files in git to keep an eye on if there were any changes outside of what I touched (spoiler, they did appear, but not immediately so with a little bit of tracking and patience I would gradually close the doors on them... and a git status would quickly reveal new illicit files) .

I used GPT to come up with a git command that only looks at deleted files and thus cleans up incrementally and ensures that it doesn't save more crap than it cleans up.

git ls-files --deleted -z | xargs -0 git add

I took the opportunity to install the Wordfence plugin that he recommended and scan the website, since it would be very difficult to find everything with finds, greps or seds. Suspiciously, the plugin didn't seem to be working very well: it scanned, but it didn't show the log of what it was doing, although you could send it to yourself by email... clearly that WordPress had basic functions broken...

"Golden shell, banana dwell"

While I was cleaning, I was on the lookout for new suspicious things, even if they weren't so obvious, and I came across this nice httpd.conf file:

That is, it did everything except configure Apache, according to GPT

Now I hear you, and this is where I wanted to get to in particular in this article, the level of understanding you can have about how a website has been compromised.

htaccess with bad drool 2.0

It's worth mentioning that I spoke to the GPT again about the suspicious htaccess files, to try to fix the Wordfence plugin a bit and for the second time they did not consider them dangerous, so until now, first the o3 model and now the o4-mini-high , failed this detection.

In this case the htaccess file was more generous, it didn't allow you to use just any php or python, it left a very long list of common WordPress administration files in the whitelist to leave WordPress half-functional, of course, to which you installed a new plugin with its own administration scripts, this htaccess that it disguised, failed and showed its brass.

<FilesMatch '^(index\.php|wp\-login\.php|wp\-cron\.php|…|plugins\.php|…|media-upload\.php|…|system_log\.php)$'>
Order allow,deny
Allow from all
</FilesMatch>

The hack is well-made, at least to go unnoticed by non-expert eyes.

The cleaning was progressing, I enjoyed having to delete more than a dozen illegitimate administrator users in the WordPress panel, Wordfence was turning off the tap and there was more or less a clear pattern to the infection: it usually generated a new folder of the same name inside a folder named ZZZ with an htaccess and a couple of smoky php.

Since I installed Wordfence and some other plugin already on a paranoia level: hCaptcha, forcing 2FA, etc. Git was warning me of changes beyond deleting files and since Wordfence, in particular, was doing a lot and I didn't know it, I asked for an explanation of some git diff, just in case.

With things stabilized and apparently no longer being able to get into my dojo, I went back to .htaccess, again with o4-mini-high, but this time explaining that things looked bad, that is, inducing the model that perhaps we would find something bad here.

The previous questions were informative and let's say that "OK" the GPT informed me what the htaccess did without reasoning what the reasons were behind it. And I put a lot of emphasis on the word reason, because at all times I use "reasoning" models that clearly... do not reason, or at least not in the direction I would like unless I make it explicit.

Oh, I also ask you to search the internet. Super important.

In most of my WordPress directories, an .htaccess file has appeared containing this line:
<FilesMatch '.(py|exe|php|PHP|Php|PHp|pHp|pHP|pHP7|PHP7|phP|PhP|php5|suspected)$'>
Can you find out the reason for this on the internet?

Result:

Having touched the feet, it now turns out that the exploit has names and surnames.

So to finish, I asked for a script to load all those htaccess files for me safely and recursively.

With this the plugins started working divinely, obviously, and the original “order deny…” error disappeared, it wasn't a problem of updating the WordPress theme.

Mission accomplished

Everything clean, no illegitimate users, the correct Wordfence scan, a few blocked IPs and the firewall already working well in automatic, we update all the credentials that are here and there, administrator users, ssh, database, WordPress hash chains, 2FA for everyone, hCaptcha, all the plugins and themes updated and with the automatic update check and a few days watching the logs and nothing suspicious appears.

It's been a lot of work and more could be done, if you know about the subject, don't be shy and comment, but the hosting is quite limited, don't come to a mega AWS solution that costs 60 bucks/month because they won't buy it from me...

Lessons learned

Let's summarize what interests us, where the LLM has helped us and where it has failed.

Great care in how we communicate with the LLM

What we ask for, how we ask for it, the detail we provide and the tools the LLM uses make the difference.

With the htaccess, it failed a lot. They were clearly part of the attack and until I expressed it to the LLM, he didn't think twice and explained to me how it worked in a neutral way.

I know that's how they work and maybe I'm asking too much (there was no system prompt telling him, for example, to be aware of possible vulnerabilities in what I share with you, for example).

But this should put us on alert and remember that an LLM is super-conditioned by our instructions, and that is why they must be as specific as possible. And consider that connecting it to the internet often adds more than subtracts (unless the information on the internet is unreliable, can you believe it?, sarcasm).

Explanation of unintelligible files.

This is the part that I enjoyed and what motivated me to write the article. I found it very interesting to discover what mischief the exploit was doing, how and why. I learned a lot of things that will help me better analyze these types of problems and also defend myself better. “Running an antivirus scan” and not even knowing what it deletes, is not that enriching.

Technical support in areas where you are not an expert

I know how to administer WordPress, Apache servers, use Linux commands, and Git, but I can't think of all the possibilities, so it's very useful to have a tool that translates your need into the appropriate command.

And good practices for managing a website with WordPress

Activate all automatic updates, have backup systems, strong passwords, 2-step authentication, the minimum number of administrator users possible and if they may not be clearly visible (for example, they should not write posts), put a prefix on the database tables, put a captcha, put a firewall, run malware analysis, install the minimum you need, if you don't need it outside the installation, an attacker with execution capacity inside your server can use inactive plugins, even if it's just to hide himself more, monitor what the crons you have running in WordPress are doing, etc., etc. This is a non-exhaustive list more or less linked to what we have touched on in the article.

And to top it off...

By the way, I mentioned a long time ago that there were plugins that shouldn't even exist... I found WP-Console, which is basically a command terminal on your server, but run from PHP. Okay, there's a reason for it to exist, when you don't have SSH access and such... but really the risk of having this there... tell me it gives the attacker a lot of leverage, in fact I don't have all the answers whether it's a plugin installed remotely or came standard with some other theme or plugin, in any case... it's no longer there.

---

Did you enjoy it? Consider subscribing for more adventures and misadventures or sharing it with your circle of geeks about these things.

Thanks for reading me!