While this post discussed the breaching of many different types of systems on the way to the Oracle database, one needs to keep in mind that the end point of any computer security breach is data. All the roads lead in one way or another to the database. While putting on barriers in every junction is important, one much not neglect to protect the database itself.

The next post will discuss how everything converges back to the Oracle database and the security measures required to protect it.

DBAs and Privileged users
DBAs and privileged users have all the access they need to modify, steal, or manipulate the Oracle database. They also have all the tools and skills required to do so efficiently and without detection.

Internal users with database access
Internal users with database access have a user and password to connect to the database. For the most part, those credentials will give them access to sensitive information. However, the tools they usually use to access the database will not naturally lend themselves to abuse. Such an individual will therefore need to

• Obtain a tool that will allow them to achieve what they are trying to do. Toad will be a simple tool, but SQL*Plus is likely already installed on their system
• Figure out how to write some SQL statements to accomplish the job. The internet is filled with examples on how to do these things.

If the credentials they have do not give them access to the sensitive data, they will also need to do one of the following:

• Borrow (with or without permission) a co-workers credentials
• Find a website with hacks and try those recipes one by one until they find the one that works. See Local Breaching below.

Internal users with system access
Any individual that can log into the operating system that runs the Oracle database, is a potential threat. The same applies to users with access to the storage and network that the Oracle databases use. Backup tapes are a little different in that anyone with physical access to those is considered a risk.

If these individuals do not have direct access to the Oracle resources (e.g. a Unix Administrator), they will need to obtain such access. As discussed above, borrowing credentials or using hacking recipes are the most obvious routes to take. See Local Breaching below.

Additionally, these individuals will need to learn how to convert the access they have to steal or modify sensitive information. For example:

• If you gain access to the Oracle Unix account, you need to login to Oracle (“sqlplus / as sysdba” will work in some environment). You than need to locate the tables with sensitive data (sometimes by joining multiple tables), and change / steal the data.
• If you have access to the storage system that contains the Oracle database, you could create a snapshot of the database, mount it on another system, start the database, locate the sensitive information, and extract it.
• If obtained the backup tapes of the Oracle database, you will need to restore those on a different system, start the database, locate the sensitive information and extract it.
• If you have access to a network that handles Oracle database traffic, you could hijack a SQL*Plus connection made by one of the DBAs and use it change / steal data.

For obvious reasons, I did not go into too much details on how to convert system access into a security breach. However, people with the right technical skills can do this, and their main challenge will probably be the lack of Oracle database expertise to accomplish the Oracle part of the breach.

Users with application access
Users with application access can either abuse their access to steal / modify information they have access to or they can steal / modify information they do not have access to.

Abuse of access by application users can be, for example, a customer call center representative stealing the information of every customer they talk to, or an HR administrator modifying the salary of her boyfriend who works in another department.

A more disturbing problem is when an application user finds a way to steal / modify records they are not supposed to. SQL injection is the most common way such breaches occur. SQL injection is a type of bug in the application that allows the user of the application to do things they were not supposed to be able to do. While searching for such flaws can be done maliciously, they are sometimes discovered on accident by a knowledgeable user.

For example, our customer call center representative talks to Mr. O’Connor, and searches for the customer information by last name. He notices that he gets an unexpected SQL error from the application when entering O’Connor. This can clue the user to the fact that the application converts the field into a SQL without using bind variables and without validating the field content. Basically, the application code looks something like:

"select ... from customer_info where last_name = '" + field + "'"

and entering O’Connor creates an invalid SQL:

select ... from customer_info where last_name = 'O'Connor'

Knowing this, a simple SQL injection can be performed by search for customer last name of

' or 'a'='a

Which will generate this SQL:

select ... from customer_info where last_name = '' or 'a'='a'

The condition of this SQL is always true and will therefore retrieve information about all the customers in the database.

A lot can be done with such vulnerabilities in the application, and a large application is almost guaranteed to have such mistake introduced by a careless programmer. SQL injection is one of the most troubling security vulnerabilities and is especially problematic with applications that are open to the internet. Part 3 will discuss measures to help with this problem.

Users with code access
An application developer or anyone with access to the code running against the database can breach sensitive information. The most famous example of this type of breach is known as Salami Slicing or Penny Shaving: when interest is calculated, there are always fractions of a penny that are rounded down. A malicious developer can accumulate those into his own account thereby embezzling funds without affecting the bank’s bottom line.

Another famous security problems involving code is that backdoor – a loophole left by developers that allows them to gain access without authenticating by regular means.

There are an endless number of examples of what developers can put in their code that will allow them to breach sensitive information.

Internal users with or without computer access
Users that work for the company have a higher opportunity of breach security than those outside of it. There are many opportunities that they can take advantage of, depending on the environment they work in:

• There are computer terminals around, and they might come across and unlocked terminal from time to time. They can also install key loggers and other security countermeasures.
• There are network ports around them all the time. Using those allows them to tap directly into the network bypassing any corporate firewalls.
• They might know other employees that will give them access (intentionally or unintentionally).
• They have access to the work environment of other employees in which they might be able to find written passwords.
• They have knowledge of the corporate structure, corporate mentality, key personnel and some procedures and policies. Such knowledge can help obtain access by manipulating individuals.

Once connected to the network, the next challenge is to obtain access to the data. To do so, one would need to breach the Oracle database, the application, the operating system, storage etc. The previous part of this post contains information about breaching these systems as well as the sections below on Local and Remote breaching.

External threats
External threats have several basic initial attack vectors:

• In many cases, companies have an internet facing application that allows customers or other individuals to access information over the internet. Vulnerabilities in these applications such as a SQL injection is the first thing an attacker would look for.
• The corporate network is protected by the corporate firewall. While beaching the firewall directly can be a problem, the firewall often times have holes allowing access to internal systems. Those can sometimes be compromised to gain access to the network
• Most companies have a wireless network (WiFi) nowadays. WiFi networks with little or no security can be easily breached. Even secured WiFi networks can be breached, most commonly by simply asking someone for the password.
• Physically accessing one of the facilities connected to the corporate network is another means of breaching security. Once inside the doors, there are many opportunities similar to those available to most regular employees.
• Calling the helpdesk or otherwise manipulating employees of the company to give access is probably the easiest means of breaching security. The human element is the most susceptible attack vector. For example, gaining access to VPN can sometimes be easily achieved by asking the helpdesk to “reset your password”.

Once connected to the network, the next challenge is to obtain access to the data. The rest of this posts has information about that.

Local Breaching
Local Breaching is the attempt to gain elevated privileges in a system that you can already log into. This can mean to obtain root privileges on a Unix system from a regular user, to obtain DBA privileges in an Oracle database from a non-DBA account etc.

Local breaching is accomplished by taking advantage of bugs in the system in question. While one might consider this to be the job for an expert hacker, most systems can be breached by following recipes.

There are many websites devoted to the collection of recipes for breaching various systems. Those recipes are contributed by various hackers that find vulnerabilities and means of exploiting those. The recipes are usually designed to be a simple matter of cut-n-paste or following a simple procedure.

You can search the site by the type of system you are faced with and find a list of these recipes exploiting various security flaws. Each recipe is usually designed for a particular version (or patch) of the system, but most users just try these exploits one by one until they find one that works.

In my experience in system security, I’ve often encountered transcripts of users that were clearly very novice, running what seemed like a complex exploit they pasted into their terminal (often having problems pasting it correctly).

As a good rule of thumb, one should assume that a user that is able to log into a system, can (with enough time and effort) obtain access to anything one that system.

Remote Breaching
While local breaching is usually a matter of searching for the right recipe, remote breaching is significantly more difficult. Most systems have little or no remote exploits and those that do tend to patch those quickly and efficiently. Searching hacking sites for remote exploits can be done, but few (if any) results are likely to be found, and none of which are likely to match the exact version of the system in question.

Remote breaching is therefore based initially on obtaining a login to the system. Trying default passwords, key loggers, network sniffing, connection hijacking, spoofing and a myriad of other tricks can be employed, including the manipulation of employees to reveal or reset passwords.

The first step in security analysis is risk assessment, so part 1 will focus on:
Who poses a risk to information in the Oracle database?

This question has the regular list of usual suspects: internal threats, external threats, privileged users etc. However, in order to be able to move forward with our analysis, we will try to be a little more specific about this list.

Part 2 discussed How will the breach occur.

DBAs and Privileged users
Some view the DBAs as the highest risk to the Oracle database because they have the most opportunity. In reality, most DBAs are honest and are offended by the suggestion of embezzlement or theft. Monitoring DBA activity tends to provide sufficient deterrence so that even DBAs who are tempted to cross the line, would not do so.

Internal users with database access
The majority of database access is done via applications and as a result most individuals that have access to information do not have direct database access. There are, however, some cases where individuals have direct database access. Such access could be granted to use data mining tools, reporting tools, or to analysts that need the access to perform their job duties.
The problem is that while these individuals have more limited database access than DBAs, they tend to have just as much access to sensitive information. Overall, I would rate their risk to the Oracle database as higher than that of the DBAs because of two reasons:

• If there are individuals in the organization with direct access to the Oracle database, this group of individuals tends to be larger than the handful of DBAs.
• DBAs have had unlimited access to information throughout their career and are used to it. This might not be the case with other individuals that have direct database access.

Internal users with system access
There are system administrators, storage administrators, backup administrators, network administrators, operations personal, and sometimes just plain users with access to the operating system, storage, network, and backups used by the Oracle databases. Their credentials might give them direct access to database resources or just to the systems that those resources reside on. I group all these individuals together because they have this in common:

• They have certain access to systems that the Oracle database uses
• They might not have the Oracle knowledge to abuse their access

Users with application access
There are probably many individuals inside and outside the organization with indirect access to data. This access is done via an application that restricts what can be done by each individual. Unfortunately, this access can be abused by various means that we will discuss in the next posts in this series.
The risks posed by this group vary significantly based on the application and the user base. Some applications are internal to the organization and protected by the organizational firewall, while others are open to the internet with less restrictions. The ability to vet the users of the application also depends on the type of application.
While the motivation of this group as a whole to abuse access tends to be higher, the opportunity they have is more limited.

Users with code access
No application is written by itself, and as such there are technical people that build and maintain it. While some of these individuals might have direct access to the data as well, they all have indirect access to the data through the code they work on.

Internal users with or without computer access
There are many individuals inside the organization the have no official access to the application or the database. However, being inside the corporate firewall, there are fewer barriers between them and the data we wish to protect.
Some of these individuals have computer access and the time to try and abuse it, while others don’t even have a computer to work on. I still group all these individuals into a single group because they posses these qualities:

• They spend every day (or most days) in an environment that has network ports and computer terminals.
• The were not given access to sensitive information and need to go to certain lengths to try and obtain such access.
• They might be aware of the information that exist in the company databases.
• They have some knowledge of the corporate structure, know other individuals in the organization, and have access to the work environment of other employees.
• They tend to be under the radar when it comes to information security as they are not typically listed as a threat.

External threats
This is the infamous group of hackers that for whatever reason attempts to gain access to sensitive information. The unique thing about this group is that in most cases they don’t distinguish between your company and the one next door.
I have already devoted two blog entries to this group: Hackers and Hacker defense. I will touch on this subject a little more in future posts.

Luckily, this is not a complex thing to achieve if you know how

The idea is pretty simple – create a java package that will open an xterm to your desktop. You will need to have some X windows server running on your desktop. Unix/Linux come built-in with X windows, and Microsoft Windows has plenty of free X servers (like Xming).

This SQL code creates the a Java package called JTERM that will run the xterm command to the specified display. Make sure you put the correct path for xterm in the cmd[] string below:

create or replace and resolve java source named "jterm" as
import java.lang.*;
import java.io.*;
public class JTerm
{
    public static void jterm(String disp)
    {
        String cmd[] = {"/usr/bin/xterm", "-display", disp};
        try {
            Process p = Runtime.getRuntime().exec(cmd);
        } catch(Throwable e) { }
    }
};
/

create or replace procedure JTERM (display varchar2) as language java
name 'JTerm.jterm(java.lang.String)';
/

In order to run this Java package, you will need a little more privileges than the regular DBA privileges. To obtain those, run this code after changing MYUSER to be your Oracle account name:

declare
  p dbms_jvm_exp_perms.temp_java_policy;
  cursor c is select 'GRANT', 'MYUSER', 'SYS', 'java.io.FilePermission',
                     '<<ALL FILES>>', 'execute', 'ENABLED' from dual;
begin
  open c;
  fetch c bulk collect into p;
  close c;
  dbms_jvm_exp_perms.import_jvm_perms(p);
end;
/

Now all you need to do is use the JTERM package to open an xterm terminal to your display. Make sure your X server accepts remote connections (e.g. xhost +), and run the package (change mymachine to be your desktop’s machine name or IP address):

exec jterm('mymachine:0')

One of the nice little benefits of this trick is that since there was no actual login to the Unix machine, there is no log of this activity in any of the Unix or Oracle logs. If you think this is a security risk and wish to monitor it, you will have to use a product like Core Audit

The drives contained unencrypted audio files of over 1 million customer support calls totaling 50,000 hours of conversation, along with 300,000 screen captures of the monitors of the BlueCross representatives at the time. To identify the individuals whose data was compromised the data had to be examined manually. It took 500 full time and 300 part time employees working two shifts six days a week.

Over a year later, on October 29, 2010, the process was complete. 1,023,209 members have been identified and notified. The cost of the breach was over $7 million dollars.

To avoid future problems, BlueCross BlueShield decided to encrypt all the data at rest. In a project that just finished, over 1,000 server drives, 6,000 workstation drives, removable drivers, recordings, and backup tapes have been encrypted. The cost of the project was $6 million dollars and it took 5,000 man hours to encrypt 885 terabytes.

What troubles me is a quote from Michael Lawley, vice president of technology shared services in BlueCross BlueShield. Lawley said that they encrypted all the drives in order to speed up the process. “Had we gone through the process of verifying and pinpointing each data store, we’d still be in the implementation phase for encryption.”

While encryption of the drives will prevent information from being compromised during a theft of a physical drive, there are many other ways data can be compromised. According to the quote from Lawley, BlueCross BlueShield does not know where all the sensitive information resides, therefore, it cannot protect it. The first step in securing data is being able to locate it.

While 57 drives that disappear are easily noticed, will BlueCross BlueShield notice if someone copied their database?

The first step in any investigation is knowing that it happened.

It is interesting that everyone is a potential target these days, and small business like restaurants do not have the skilled man power needed to properly defend their computer systems.

The article also had some quotes from Jerry Silva, a consultant in the financial services industry: “In the end, compliance with the Payment Card Industry Data Security Standard is the best way to prevent cardholder compromises. The problem, however, is that many merchants and processors remain out of compliance… Sometimes, if you are a merchant acquirer and are showing a good faith effort to get PCI compliant, a lot of times the auditors will let it go. If they are making good progress, then the auditors sometimes will be lenient. Compliance does not always mean compliant.”

It’s hard to create security through regulations, and PCI-DSS is a good attempt. Unlike most compliance regulations, PCI-DSS provides reasonably detailed technical requirements and not general concepts about analyzing risks and mitigating them. It even requires database auditing explicitly, which I consider crucial.

Another quote from Silva later in the article is “It’s almost like we need a different model, like federated security… The process we have in place is not working. And I don’t think EMV [Europay, MasterCard, Visa standard] will solve it. I think EMV does solve some of the issues, but not all.”

I have to say that I agree. The problem in these cases is no longer knowing what to do, but actually doing it. Yes, databases should be audited. There is plenty of evidence to support this. But are YOU auditing your databases? Will your auditor let you slide because you’re making an effort? While I’m all for helping out the little guy, I’m not sure I would be a happy customer if the restaurant I ate in last was breached.

Requiring a restaurant to be PCI-DSS compliant is ridiculous and will never happen. Not requiring a restaurant to be compliant or cutting corners will compromise credit cards and make the unfortunate customers very unhappy. Something has to change, the questions is what. This might be a good place to end this post, but I don’t like questions without answers.

My opinion is that if you choose to store credit card information in your computer systems you have to be PCI-DSS compliant all the way, and hopefully more. Anything less is clearly a problem. But I think that the current method of credit card processing puts unreasonable burden on the small businesses that use them. Instead of storing card information in various devices along the way to the processors, credit card terminals should immediately encrypt and transport the information offsite. Small businesses have no business storing credit cards in their systems. If they choose to do that, they have to be PCI-DSS compliant.

Another way of putting this is that all POS systems should be PCI-DSS compliant. Forcing vendors of credit card terminals and various POS software to go through rigorous audits will promote an “encrypt and send” methodology. Pushing this functionality into the card readers and avoiding local storage will significantly improve security on the small business end.

I think there are a handful of good things about the actions taken by these hacker groups:

• They educate the public about the security problem in IT and the danger to their personal information
• They educate security professionals about just how vulnerable their systems are
• They educate executives about the vulnerabilities of their companies
• They educate everyone about the fact that every organization can be breached

All these educational consequences will help bring more funds, focus, and concern to the safety of data.

On the other hand are the financial, reputation and other damages caused by these actions to companies, as well as the publicity of information that might otherwise remain confidential.

To add more to this dilemma of good vs. bad, you need to remember that there are other hacker groups as noted in this post that are pure criminals – they steal data to sell it. One might claim that all the data compromised by Anonymous and LulzSec is just waiting for the criminal hackers to steal and sell without anyone knowing.

I believe that if the education provided by Anonymous and LulzSec prevents criminal hackers from breaching some of their targets, it is a worth while education. The reason is simple – criminal hackers go after information they can sell to obtain maximum profits, and therefore, tend to cause the most harm. Anonymous and LulzSec go after information they can portray as embarrassing to those they stole from, and do not seem to go after information like credit cards and social security numbers.

In my humble opinion, the biggest security problem in most organizations is that they would not know if they have been breached. This matter is explained in detail in this post. Without knowledge of a breach, massive volumes of data will be compromised, like here.

Had we known the number of breaches in the last two years vs. the number of breaches in the next two years, we could attribute the change to the public stunts of Anonymous and LulzSec. Unfortunately, we will never know about many breaches, and the question of good vs. evil will remain a matter of estimate and opinion.

I found the results to be very interesting. The survey shows concern is mostly about inside problems. The further outside the threat is, the less it is considered a threat. I find this interesting because many security people tend to be worried about external threats that are not well defined. I suspect the reason for worrying more about the internal threat is the size of the organization and an inability to control it.

Insider threat and poor practices rank at the top of the What list. This demonstrates, in my opinion, a lack of control over the organization. The next item on the What list is Exploitable software vulnerabilities. Software vulnerabilities are a bigger threat when there are many software packages from many vendors deployed across the organization with little control. As you can see, everything points to a large organization that is not security minded.

The Who list also shows a similar trend as Careless users rank number one followed by Inside employees and Inside contractors. The trend is clear when moving further outside the organization the threat diminishes.

The question I pose after reading such a survey is “What should a security professional do in such an environment?”

One path is the political path. Push for more support from management. Management support will bring security mandates, more security people, more budgets, better training and a general focus on security. It would be great to have such a security focus from management, but it will take time. Given the way the government work, it might never happen at all.

The second path is more active – Focus on the resources you need to protect, and guard them from the company. The problem in protecting the data from the company is that preventative controls cannot be effective. The employees that are considered a threat need access to the data. The systems that are considered vulnerable need to be used to access the data. So while preventative control are important, they are far from being sufficient against an internal threat.

The only solution I know to this problem is Activity Monitoring. Watch who’s doing what in your systems in general, and in your databases in particular. Database Auditing / Activity Monitoring is a fundamental tenant in any security strategy, and is the only security measure against insider threats.

See my blog entries that analyze audit reports of the DHS and the IRS that arrive at similar conclusions.

“Diary of a Breach” was a well written piece that walked the reader through an imaginary breach scenario. Was caught my attention was the fact that in that story the DBA was able to go into the database and figure out exactly what happened. It was a very impressive piece of investigation that cannot happen in real life because that information just doesn’t exist. A database auditing tool has to be deployed for that information to exist, and most organizations do not have one.

It reminds me of movies that show a genius programmer hard at work and all you see on his screen are HTML fragments. Older movies used to show lines of Basic, but the principle remains. It might look impressive to most people but if you understand what you’re looking at, it’s just hilarious.

You’re probably wondering what all that has to do with the SAFE Data Act. The article I was reading in Gov Info Security explained how the new law would require companies to report breaches within 48 hours and proponents took issue with the vague definition of what information is covered by the law and whether anyone will actually enforce this.

I’m skipping a lot on an important discussion because everyone forgot the beginning of the story – How will a company know it’s been breached?

It took TJX 18 months to know it’s been breached. And if Anonymous hadn’t claimed it hacked Booz Allen Hamilton, that breach might never have been discovered. It was a little funny to read how all the articles noted that Booz Allen Hamilton hasn’t confirmed the breach when a dump of their database was available for download on the internet. It’s not simple to know your systems have been breached and even harder to know what happened.

While some Hackers publish their victories to claim fame, others do it in secret to get paid for the stolen information. I suspect many companies are breached without ever realizing they have been. That is why the discussion about the notification procedures of breaches is pointless without being able to detect them.

Without reviewing database activity on a daily basis and having full knowledge of who has done what, you will probably never know a breach occurred, let alone what was breached.

For those that don’t know what the FFIEC is, it is The Federal Financial Institutions Examination Council (FFIEC). The FFIEC was established by Congress in 1979 to prescribe uniform principles, standards, and report forms for the federal examination of financial institutions, to make recommendations to promote uniformity in the supervision of financial institutions, and to conduct schools for examiners.

Guidance offered by the FFIEC is to be followed by financial institutions and is enforced by the examiners the FFIEC trains. It, for example, FFIEC would require databases to be audited, financial institutions should follow. So I dug a little through the FFIEC guidance to see just how explicit the requirement is. While I’m certain that there are many pages in the guidance requiring database auditing, here are a few that I found.

Database Management is a very short page dealing with databases and covers some basic security principles. If you work in a financial institution and deal with databases or security, I recommend spending 5 minutes to read this page.

First, I would like to correct a statement made in that page: “It is possible to control, monitor, and log access to data … but there is a systems performance cost.” Core Audit provides Full Capture of all database activity at less than 3% overhead. The Full Capture technology developed by Blue Core Research is the only one that allows for such low overhead, so there is some truth to the statement. But while other tools would impact system performance, It is not an imperative.

The following quote from the same page explicitly requires monitoring of DBA activity via a database auditing tool:

“The primary risk associated with database administration is that an administrator can alter sensitive data without those modifications being detected. A secondary risk is that an administrator can change access rights to information stored within the database as well as their own access rights. As a preventive control against these risks, the institution should restrict and review access administration and data altering by the administrator. Close monitoring of database administrator activities by management is both a preventive and detective control.“

The page about Database Management Systems also explicitly requires database auditing:

“organizations should employ automated auditing tools, such as journaling, that identify who accessed or attempted to access a database and what, if any, data was changed.“

Another page requiring database auditing is about access rights:

• “Formal access rights administration for users consists of four processes: … A monitoring process to oversee and manage the access rights granted to each user on the system.”
• “Authorization for privileged access should be tightly controlled. Privileged access refers to the ability to override system or application controls. Good practices for controlling privileged access include: … Logging and auditing the use of privileged access …”
• “Default user accounts should either be disabled, or the authentication to the account should be changed. Additionally, access to these default accounts should be monitored more closely than other accounts.”

The last statement clearly suggests that while certain accounts should be monitored more closely, all accounts should be monitored.

The Activity Monitoring page is focused more on host and network activity monitoring, but has short list of security events that applies to databases as well: “Examples of security events include operating system access, privileged access, creation of privileged accounts, configuration changes, and application access.”

I honestly think the FFIEC guidance couldn’t be any clearer, but to understand the value of activity monitoring have a look at the Security Monitoring page.

The report discovered what I would consider fundamental security problems in the IRS databases, but I found other information in this report to be more interesting.

TIGTA has many things to inspect and the fact that they chose to inspect database security in particular for the second time in a few years shows they consider it important. To quote from the report:

“Databases are increasingly being targeted by attackers. A 2009 report on data breaches cited that 30 percent of all known security breaches were against databases. This trend was particularly disturbing because when a database was breached, 75 percent of the records were compromised.”

Another interesting paragraph from the background of the report is this:

“The Internal Revenue Service (IRS) employs almost 100,000 employees and operates more than 200 applications to administer our Nation’s tax laws and regulations. The IRS relies on more than 2,200 databases to manage and process data, such as personally identifiable taxpayer information and sensitive financial/tax information, on its computer systems. Two database management software products are primarily used in the IRS’s non-mainframe computer processing environment: Oracle and Microsoft Standard Query Language (SQL) Server.”

When reading this paragraph I can’t help but wonder how does one secure 2,200 databases used by 100,000 individuals? Can anything be secured in such a scale? Maybe more interesting are the ratios as it appears that on average every 45 employees in the IRS have their own database and every application in the IRS requires 11 databases to operate.

Finally I enjoyed this piece of information that repeated in multiple places in the report:

“The IRS spent more than $1.1 million in software licenses and support costs for a tool that was not fully implemented.”

The reason I enjoyed it was that it was said about one of our competitors. Another part of the report referring to this issue says that:

“IRS management explained that they experienced significant technical difficulties in implementing…”

I know that Core Audit is significantly easier and more powerful than any of the competing products as I constantly hear it from customers. It’s also no surprise to me that the IRS has failed to deploy this tool and will likely fail to deploy other competing tools. I was only surprised to see it so bluntly pointed out as one of the main findings of this report.

But going back to the size of the IRS environment I can’t help but wonder – how will anyone know if there was a breach? With so many databases and employees, is anyone keeping track of who’s doing what? Without database auditing deployed and used across all the IRS databases all of our personal information is out for grabs.

The audit found some issues with the security of PCII (Protected Critical Infrastructure Information), but I’m honestly more troubled by another issue. Reading through this report I couldn’t help but notice that every single problem was a violation of one policy or another. For example “These accounts were configured to be disabled after 90 days of inactivity. Under DHS policy, unused accounts should be deactivated within 45 days.”

Which means that the entire audit process was dealing with just how well DHS follows its own policies or policies it is subject to. It’s bad to know that DHS doesn’t follow its own policies, but I’m a little more concerned with the security situation that is outside the definition of DHS policies. I’ve been dealing with security for many years and have found that policies are not what you might consider good security. Policies are not bad to have, and even better to follow, but they are a small fraction of any good security. Maybe I should not expect a lot from the government, but I hoped that a security audit of the DHS by the DHS would include more than checking the published manual.

There’s another subject that I found very troubling. Have a look at the following quotations from the report:

• Seventy-two of the 4,807 active ACAMS users have never logged onto the system. Other users had not logged onto the system for almost 5 years”
• “Eighty-three percent (4,005 of 4,807) of the active ACAMS users had not logged into their accounts for more than 45 days prior to the date the list of users was pulled for testing. Four of these accounts had “super user” access, which grants unrestricted administrative access to ACAM”
• “Four ACAMS website administrators have duplicate unrestricted access to the system; each administrator has two “super user” accounts.”
• “Thirty-seven local administrators have privileges on the ACAMS servers.”
• “Thirty-three local Windows administrators have been granted access to ACAMS’ PCII database. The SQL database settings allow all local Windows administrators access to the PCII database.”

After reading this, I couldn’t help but wonder how on earth can the DHS secure information that almost 5,000 people have access to, 70 administrators, and a few other floating “super user” accounts. Does anyone monitor what all these people do? Will anyone know if a password was stolen and used to access this information?

I think that based on the fact that 72% of the users have never even logged in, there’s no human inspection of the privileges or the activity.

Application and database activity has to be monitored. There is no security without it. You cannot protect a system without knowing what’s going on inside it. Will anyone know if someone obtained a password to these databases, logged in, and dumped all the data out? With thousands of people that have access and dozens of administrators with privileged access, even the disgruntled employee scenario is not a far fetched idea.

I think the only reason we don’t know of a DHS breach is that no one will know if there was one.

The July 2011 issue is titled “Threats in the Supply Chain” and is about the security risks incurred due to business relationships with other companies and individuals. The first half of the issue demonstrates through recent attacks that business relationships between companies pose a security threat to the organization. I completely agree with this matter and had several posts about it, including Security is an Illusion.

The second half of the issue is an article by Robert Lemos that makes a 4 point suggestion for a solution:

1. Document Vendors And Their Policies
2. Minimize Access And Permissions
3. Trust Nothing And No One
4. Protect The Data

The first two steps are standard security measures. The 3rd step is about admission that perimeter security such as a firewalls is not a good enough defense in a world where many individuals and companies are granted access through it. The conclusion is to secure and monitor the internal environment and not just the perimeter. Lemos provides little detail about this, but I suspect most readers know what it takes to secure an environment. The main point is not to assume that a firewall will hold back an attacker, and monitor and defend what is usually the soft insides of the organization.

The 4th step is, IMHO, one of the most important points security people need to understand. The article is lacking in clarity, focus and suggestion about this subject so I will add my own point of view to it:

Organizations are very big and often contain tens of thousands of computer systems including desktops, laptops, network devices, servers and much more. Due to this scale, it is almost impossible to secure each and every single system, and consequently, security focuses on the perimeter. Unfortunately, firewalls cannot prevent attackers from getting in and we now have to protect each and every one of the tens of thousands of computer systems.

It’s important to protect each one of these devices, but it’s not realistic to provide an efficient defense for each system. This is where step 4 comes in – Protect the data. It’s true that breaching several desktops in your organization would be bad, but it would pale in comparison to a breach of your databases.

I think that any security person needs to start by asking two basic questions:

• What information am I protecting?
• Where does that information reside?

All the security measures you deploy eventually boil down to protecting this information that resides in these locations.

Lets move from theory to reality. The majority of the information you’re protecting resides in specific tables in certain databases. I’m not a mind reader, it’s just the reality in every organization nowadays.

Physically that information resides either in your own data center, in an outside data center, or in a cloud. If physical access to the information cannot be controlled, the information has to be encrypted.

Next are the network pathways to that information. There’s direct database access or access through application/web servers. Each of these needs to be secured with access control, firewalls, encryption, etc.

Unfortunately, just like the perimeter defense, these defenses can also be compromised. The next line of defense you need to deploy is activity monitoring (aka auditing). This mainly falls into two categories: database auditing and application auditing.

I naturally think this is important because Blue Core Research makes an Oracle database auditing tool called Core Audit. It can audit all the activity in the Oracle database and even monitor certain application activity. While I might not be the most objective security person on this subject, I honestly don’t know of any other measure to protect the data.

The longer I spend in this business, the more it seems to me that auditing is the most important measure. The reason is simple – it’s very difficult to penetrate a database without a valid user and password. Most standard defenses you probably have deployed will almost guarantee this. This leaves one very difficult problem – How to protect against attackers that obtained a valid user and password. Without database auditing such an attacker would be completely invisible.

This is important to me because my personal information is floating out there in various databases. Mine, yours, and everyone else. All it takes is a single DBA in the IRS that uses his IRS user and password for something like his personal email. It doesn’t even have to be a DBA account that is compromised as anyone with access to any database that contains my information poses a personal security risk for me. And if that’s not enough, anyone those companies do business with also poses a security risk for me.

So please protect your data. It might be my data that you’re protecting.

I should first start by saying there is no fixed recipe for defending against hackers. There are many things you should do to secure your organization, but hacking is a type of out-of-the-box thinking that does not follow the security conventions you are following.

So I started thinking back to times when I wrote software that was supposed to be hacker resistant. In this post I’ll take you through a 3 phase process of trying to defend against hackers. To successfully finish this process you will have to use both your logical thinking and your emotions. This is the only way to get closer to defending against hackers.

Phase 1 is pretty simple and you are hopefully doing this already. Setup the conventional security measures that you know you need. Firewalls, non-default and hard to guess passwords, minimal privileges, encryption and so on. You don’t need my help to do this part.

Phase 2 is the logical step. Think about how you could break into a company. Here are some ideas:

• Look for an open wifi
• Call the helpdesk and say you can’t log into the VPN
• Walk in the door and plug into the network. If there’s access cards you can probably tailgate
• Scan the network for open ports
• Dump the DNS and look for suspicious machine names
• Sniff the network for passwords (yes, this can be done in a switched network too)
• Hijack a connection that is already logged in (again, switches don’t prevent this)
• Try to walk into the server room
• Go to where the admins sit at lunch time and look for a password list, backup tapes, used drives, open laptops, or anything else you can use
• Use an open terminal of someone that just went to the bathroom (or to lunch)
• Call the helpdesk, pretend to be John Doe, and ask them to reset your password

I just spent 5 minutes thinking about this. I’m sure there’s lots of other things you can think of. Obviously, you’ll need to find a way to close these holes somehow. So far we came up with the simple things. They are floating around in your mind and you know them.

Phase 3 is about pulling all the stops. You might not realize it, but you’ve been holding back. To pull all the stops you need to get emotional about the subject. You’ll need to imagine you truly hate the company and you want to cause them irreparable harm. For example, on Friday afternoon imagine you’ve been fired for something you haven’t done, escorted out of the building in front of everyone, and the company is slandering your reputation. I’m not sure if this story will work for you, but I’m sure you can think of something creative.

The point of the exercise is to spend the weekend dreaming up how you’ll get into the company and cause damage. There are no boundaries. For example:

• You know someones password
• You know an application password
• You know of a back door into a system
• You have a friend that you can ask for a favor to get you in
• You know that if you send an email to the receptionist with a key logger saying she won the lottery, she’ll open it
• You know that one of the admins will not think twice about trying to open an attachment that is supposed to have a joke
• You know the smokers sometimes leave the back door open so they can come back in
• The annoying password policy of coming up with a new password every month got people to use their first name followed by the month number

If you can’t come up with ways to penetrate your company, it means you’re not trying. What you are wondering is how can a hacker know all these things you know. After all, you had to work for the company for a long time before you found all this out.

The simple answer is that the person you might need to defend against is not a hacker but a disgruntled employee (or ex-employee in the previous example). The more complex answer is that sometimes hacking involves a lot of intelligence work. The information you collected accidentally over several years can be collected deliberately over several days with enough motivation and some social skills.

You should now have a list of things you need to defend against. The reason almost any place can be hacked into is that this list is not simple to defend against. Remember that if you cannot prevent a hack, the next best thing is to know it happened and mitigate it. Many systems, Oracle databases included, cannot be fully protected, and require activity monitoring tools such as Core Audit to detect breaches.

First I think it’s important to understand that hacking means many different things. It can mean to penetrate a network, to escalate privileges on a system, to trick a system into doing something it’s not supposed to, to break software licensing, and much more. It all depends on the context the word hacking is used in.

The reason I’m stressing this is because each type of hacking affects different types of companies. A software vendor might care about their software licensing being circumvented or their software security being compromised. Other companies will care about their website, while others still will care about their network being penetrated. These are all different types of hacking by different people with different motivations and skill levels.

secondly, I think it’s important to note that there are different motivations for different hackers. For example:

• Hackers that do it for fun and ego – This group does not intend to cause serious damage. They try to get into places that are difficult to demonstrate their skills, or to do minor damage that they consider funny. Saying your systems are unbreakable gets the attention of this group, and they will find a way in. The targets of this group tend to be circumstantial and sometimes pure random.
• Hackers that do it for revenge, anger, political reasons etc – This group is out to do damage. They target specific companies that they feel to be “evil” and try to cause as much harm as possible.
• Hackers that do it for financial reasons – I consider this category to be common criminals. While anyone with a skill wants to get paid for it, abusing a skill to make money illegally makes you into a criminal. They target personal information, credit cards, or any other information they can sell.
• Hackers that do it as a job – Another way to make money from hacking is to get hired by a security company, government agency and similar positions. Morals aside, this is simply a job. They target whatever their employer needs them to compromise.

The motivations are important to understand because knowing who you’re up against is a good start. If you work in a high profile company that has a reputation for security, you will likely attract the first group. They are not out to do too much damage but they will definitely make it very public and as humiliating as possible for you.

If you work in a company that stores large amounts of personal information in your databases, you are a target for the criminals. They don’t care about publicity – they are just after the data. Getting in and changing the home page on your website is not as interesting as dumping the content of your databases. They would prefer if you will never know they got in.

If you guard top-secret materials, you are a target for foreign governments. Like the criminals, they are not out for publicity. They want to get in, get out, and leave no trace. Since they tend to have significant resources and training, they are likely to be very good at being invisible and leaving no evidence of their visit.

The third thing to understand is that there are different levels of hackers. The more advanced the hacking skills, the smaller the group and the more likely it is to break through your security.

• Download! – This group knows little about security or how to compromise it. They just download hacking tools or hacking manuals and follow the steps. Defending against this group is mostly about dotting the i’s and crossing the t’s. This group is the largest by far, it looking for recognition of achievement, and is likely to go after any target. Vulnerability assessment tools might help, but only against this group. While such tools will not prevent a breach, they will help you cover your bases. Having the latest patches is also important.
• Trial and Error – This group is good at software testing. They look for a bug and then look for ways to exploit it. They can manipulate URLs, Do SQL Injection, enter bad information in various fields, click on things in the wrong order, enter random license keys, and more. Since your applications are likely to have bugs somewhere, the only defense against this group is to prevent access to your applications.
• Engineering – This group usually obtains the software you have and “test”s it. They use a variety of tools to trace the program execution and understand its interaction with the environment. They use this knowledge to locate vulnerabilities in the application and exploit them. Exploits can be registry entries, files, permissions, race conditions, and more. They sometimes publish hacking tools and hacking manuals for others to enjoy. The only defense is having a proprietary application that they have no access to.
• Reverse Engineering – This group usually does hacking as a job. They decipher the program through assembly or similar means and locate vulnerabilities in the code. They can find back doors, weaknesses in encryption, algorithm mistakes, key generation vulnerabilities, and more. Proprietary code is the only defense.

My best advise to anyone trying to defend their systems is to know what is important to protect and why. Just like a bank will invest less in safeguarding the pens on the counters and more in protecting the money in the safe. You must ask yourself what do you have to protect most and focus at least 60% of your resources around that.

If you have large amounts of personal information or credit cards, you are a target for the criminals and they will probably be of the 2nd or 3rd skill level. The main thing to protect are the databases that contain the relevant information and the applications that access them. Oracle auditing tools like Core Audit will help achieve this goal.

Read my next post about Hacker Defense for more.

I have never personally known anyone that committed fraud and was therefore surprised by heart felt empathy for those who suffer the consequences of their criminal actions. I kept going back and forth in my mind between feeling compassion for the people and feeling anger towards the actions they took. After all, most of us don’t commit crimes even when times are hard. On the other hard, it’s difficult not to feel compassion for people who have taken their own lives when faced with the consequences of their actions.

Towards the end of the blog, there was a single sentence that caught my eye: “Deterrents are powerful, and can help offset motivation if they are used effectively.” and I realized it is the key to resolve to my ambivalent feelings towards the subject.

I have to admit that the sentence first caught my eye since we are in the business of Oracle database auditing. Oracle database auditing has the explicit purpose of detecting fraud, theft, and breaches. Due to its detection abilities, Oracle database auditing is also a powerful deterrent.

But being in this business I also know just how easy it is for individuals with access to commit fraud or theft without leaving any evidence. It amazes me that so many are caught committing fraud and theft as it is so easy to hide any traces of your actions. I often think that the number of fraud and theft cases that occur is much higher than those we know of as I suspect many are left undiscovered.

So here is the question – if a poor man walks next to an open window every day, and every day there is a $100 bill sitting on the ledge. This man knows for sure that no one is watching him and he can take the bill without anyone knowing it’s missing. How guilty would you find this poor person if after many years of looking at that $100 bill every day he finally reaches out and puts it in his pocket?

It’s definitely a crime that he shouldn’t commit, but wouldn’t you place some blame on the owner of the money for leaving it unattended for years on an open window ledge? For those that are not in the database world it might seem ridiculous, but this is the exact situation in every database system, banks included.

I don’t want to defend those who committed these crimes. It’s fraud, it’s premeditated, and it’s committed with intent. But I have to place some blame on the organizations that do not monitor the activity in their databases and make it so easy to commit this fraud.

Whenever such cases come to light, the bank or financial institution in question always expresses their outrage at the criminal actions. And I say – look at what your organization has done to prevent these actions. Examine your internal monitoring and deterrence. Think of the temptation you dangle in front of your employees to commit these crimes.

My conclusion – both the organization and the employee are guilty in these crimes, but only the employee ends up sitting behind bars.

While I was very skeptical about the ability of the government to improve security, I now think they might be able to do so. The solution of the government to the problem is similar to many solutions of the Federal Government to problems – throw money at it. Lots of money.

Granted, it doesn’t take hundreds of millions of dollars to develop technology, and most of that money will be wasted on bureaucracy and various incompetent attempts that will produce nothing. But with enough money going specifically to cybersecurity, it is possible that some new ideas will emerge.

Having been in the software industry for as long as I have, I know that innovation rarely comes from large corporations taking federal money and wasting it on nothing. I know that the solution to almost any problem is going to come from a small startup with an innovative approach. But throwing money into the system will hopefully get the ball rolling. It’s just a shame that most (if not all) of our hard earned tax money that is being thrown, will end up doing nothing.

As I said, I started being very skeptical but now I have some hope. It’s a very mixed feeling, but there is still some hope that a solution will emerge. And I think most people will agree that it’s a better way to waste our money than giving it to the banks.

Security by obscurity is a term used to dismiss security measures that are not founded on strong mathematical principals that make it theoretically unbreakable. Don’t get me wrong – I have no objection to all the encryption and key exchanges that are based on strong mathematics. My problem is not with the theory but with its implementation.

The problem with the strong encryption and other mathematically intensive algorithms is that very few people implement those. The result is that most applications end up using the same libraries to implement such algorithms. Worse is the fact that there is a limited number of products that provide security solutions. The result is that any hacker or attacker can get their hands on the product you use to protect your environment. Additionally most of these products share large portions of the code that does the underlying math, protocols and so on.

Why is all this bad? Because every software has bugs and security holes. It’s not something we should be happy about, but it’s a fact of life. The more complex the software, the more bugs and security vulnerabilities. Complex security systems that rely on heavy mathematics will undoubtedly have their share of security holes. Since hackers can obtain the software, they can examine it until they locate those security holes. For extra incentive, once they find the security hole in the software they have hundreds if not thousands of companies they can breach.

Now imagine a world were every company wrote its own preventative measures. Maybe not the most mathematically advanced measures, but none the less unique. For hackers to break into these companies, they would need to first understand the security system, then look for vulnerabilities, and they have to do it all live. If they manage to find a way in, they still compromised only a single company. If that company is compartmentalized with different security measures protecting different departments, the hacker will need to put in a significantly higher effort to get deeper into the organization.

We don’t have to go as far as having unique measures in every company in the world to get better security for our own organizations. We just have deploy unique solutions that no one else has. It is the only way to stop hackers from getting in. Any off the shelf security solution you buy will have vulnerabilities a hacker might know.

If you can’t build your own solution, you must monitor the activity to detect breaches. In the case of databases it is impossible for you to build your own, and you must therefore monitor the activity. Oracle database auditing tools like Core Audit provide such activity monitoring and are essential to secure the Oracle database.

I recently read the Federal government interpretation of continuous monitoring and felt depressed. I read the NIST FAQ, some statements from Vivek Kundra, and an InformationWeek article by John Sankovich about the new Federal guidance. I probably shouldn’t have had high hopes about how the federal government plans to implement security.

The Federal government understanding of continuous monitoring is to have more frequent assessment of the effectiveness of their preventative controls. It’s not a bad idea, but how many of you think that will stop a hacker from breaking in? This is a small step in the right direction, but it’s like jumping in the air and thinking we are closer to the moon.

Continuous monitoring includes two other concepts that are just as important is not more so – Continuous Audit and Continuous Transaction Inspection. In the financial world these terms are reality with multiple tools and technologies that attempt to detect fraud, market manipulation, etc. Leveraging such technologies in financial systems such as trading systems is often required by regulations.

My favorite security analogy is that of a bank and I often ask potential customers whether they will put their money in a bank that has no security guards, no security cameras, no alarms… only a big vault door. A bank like that is not much different than my mailbox. The only difference is whether it will take an intruder 2 minutes or 2 hours to break the lock. But with no alarms or security guards, a bank robber will have all the time he needs.

I like this analogy because it clearly demonstrates that security is all about the people. Everyone understands this intuitively, but it is somehow forgotten when we secure computer systems. My interpretation of continuous monitoring is simply a security guard. This is not an analogy, I’m talking about a real live person. Someone watching monitors that show what’s going on in critical computer systems and reacts to suspicious activities.

The concept of an operations center (e.g. NOC) is not new, but it currently only shows metrics and anomalies detected by automated tools like ArcSight ESM. Monitoring live transaction activity in Oracle databases, networks, and so on will require a lot more people than the current NOC. True implementation of this concept also requires many tools including Oracle database auditing tools like Core Audit. While my interpretation of Continuous Monitoring requires more people and is, therefore, more expensive, it is also a powerful security measure that will be hard to defeat.

According to the post, the attack was easy and took only 4 man hours. It was easy because they managed to find a server with no security. After penetrating the network, they got passwords, sources and much more. Their best treasure seems to be a dump of roughly 90,000 military emails and password hashes that were not salted (non salted hashes allow for a much easier dictionary attack). Additionally, they found various information that will help them penetrate many other government agencies, federal contractors and various companies.

We would expect better security from Booz Allen Hamilton, and so did the hackers from Anonymous. But more troubling are the passwords and other credentials that were obtained that would allow penetration of many other networks with better security.

Hacking is like looking for a piece of string. Once you find a string, you start pulling on it, and it takes you wherever it takes you. Everywhere you get to, you look for additional strings to start pulling on that will take you to even more places. It is similar to an avalanche as every additional piece you get allows penetration of many other pieces.

There’s a common analogy saying that security is like a chain, and its strength is that of the weakest link. Unfortunately, it’s not only your chain that needs to be strong, but also that of everyone you’re associated with, everyone they are associated with, and so on.

In other words – you can be breached. If not due to your weak security, then because one of your employees uses the same password and it’s stored in another company that isn’t secured very well. And if all those are secured, than one of the employees in those companies might have a password in other places that are not secured. Sooner or later, those strings that hackers pull on will lead them to your network and they will find their way in.

If any security can be breached because hackers find passwords to your systems, there is only one option – monitor the activity of your users and look for the attack. Detecting the attack will allow you to stop it as well as handle the consequences. Knowing what was compromised and how is the most important aspect of handling an attack.

Focus your resources on the main information storage you need to protect like databases and big file servers. While hackers will always find interesting things inside your network, doing as they please inside your databases without anyone knowing is truly the worse case scenario. Blue Core Research specializes in Oracle database auditing to enhance your Oracle database security. Ask for a demo and see how we can help.

The situation is a little more absurd as Foster allegedly embezzled the money during the second half of 2010 and left Citigroup in January of 2011. The alleged embezzlement was only found in late June, about six month after Foster left Citigroup and was traveling the world. The FBI caught Foster in Kennedy airport as he arrived on a flight from Bangkok after a trip to Europe and Asia.

To read more about the Citigroup embezzlement go to the New York Times.

Naturally I got curious. A bank that was in the center of the financial crisis less than two years ago; A bank that has been forced to improve its security measures specifically against internal fraud; One of the largest banks in the United States; Such a bank takes a year to notice $19 million are missing. Yes, it’s a huge bank, but I would have expected such an institution to employ the best security measures.

The answer is painfully obvious – Citigroup was not doing a good job of monitoring insiders. As I kept reading about it turned out that this is the situation in most banks. Earlier this year, 4 banks including Bank of America and Wachovia had account and personal information of 676,000 people stolen. Orazio Lembo has allegedly bought this information from bank employees and later sold it. 10 people were arrested including Lembo and 7 bank employees.

Police statement said that “Based on forensic examination of Lembo’s computers, it was determined that he had employed upper-level bank employees to access and identify individual accounts in their respective banks. That information was then sold to his clients, which included more than 40 law firms and collection agencies.”

The amazing part is that this security breach which may be the biggest in the banking industry, has been going on for 4 years. We expect bank employees to be monitored, but they are clearly not monitored effectively.

To read more about this breach go to CNN.