Database Auditing & IDS: A Comprehensive Guide to Data Protection

Introduction

Modern businesses run on data. From customer data to financial information and beyond, databases store plenty of sensitive information. This data enables company operations and drives decision-making. However, this reliance on data exposes organizations to significant risks. Cyberattacks and data breaches can compromise sensitive information, leading to financial loss, regulatory penalties, lawsuits, and irreparable damage to brand reputation. To mitigate these risks, organizations must take action to protect their data. That means protecting access to the databases that store it.

What is Database Auditing

Database auditing, or Database Activity Monitoring (DAM), is a database intrusion detection system (IDS). It monitors connections to the databases and SQLs executed in those connections. While that might sound like a pointless exercise, when done right, it is the most critical database defense and one of the most significant security measures to protect your data.

The challenge in database auditing is the activity volume. There are billions of SQLs per month, and no one can manually review everything. In reality, the challenge starts before processing the data, as capturing this amount of activity is difficult without creating a performance overhead and slowing down the database.

For database auditing to live up to its potential and provide a solid defense, it requires a powerful capture mechanism that can see everything without overhead and powerful processing capabilities to allow you to derive value from all this data. That is a lot of technology and knowing how to use it, and that’s what this article is about.

Database Auditing: An Old Misnomer

Database auditing often brings to mind traditional, declarative methods for monitoring database activity. Usually, by turning on auditing features in the database, recording activity, and building reports. While these methods have their place, they fail to address the security challenges of modern database environments. Today, the focus is on more comprehensive Database Intrusion Detection Systems (IDS), the primary component of an advanced modern database security solution. With superior visibility and control over database activity, they enable organizations to detect and respond to current threats.

The Power of Database IDS

While intrusion prevention systems (IPS) can play a role in database security, IDS is more effective due to the unique nature of database access and the way database attacks are performed.

Databases typically have a limited number of accounts. Connecting to a database without valid credentials or an approved access method (such as a local connection) is nearly impossible. The challenge is, therefore, to control the activity that comes through these approved channels. The problem is the sheer volume of activity. With billions of valid SQLs per month, it is exceedingly difficult to identify the handful of malicious ones that might be an attack.

Preventive systems must have a low false positive rate. The lower the false positive rate, the higher the false negative rate. In databases, blocking a valid SQL mid-transaction can have catastrophic consequences, so database IPS has an especially low tolerance for false positives (and, therefore, a lot of unprevented events). The other problem in databases is to distinguish the good from the malicious activity. So, when combined with the inherent limitations of IPS, prevention is only effective for a small number of attack vectors.

In contrast, detective systems have a higher tolerance for false positives. A modern database IDS solution can help organizations gain visibility into their database activity and identify most attacks with a reasonable number of false positives.

Visibility: A Precondition

Whether you intend to deploy an IPS or an IDS, you must first gain visibility into your database activity. You cannot secure what you cannot see. It is impossible to design effective policies, rules, or reports without a solid understanding of the activity you are trying to control.

By gaining visibility into your database activity, you could:

Design controls: Understanding the activity profile will help you determine how to gain control over it. You need to identify, for example, things that happen, and you should be aware of those. Other things rarely occur, and you should alert or block them. Setting up security based on a theoretical guess of the activity is ineffective. It is impossible to secure what you cannot see.
Identify risky practices: In many environments, users and administrators have adopted practices that are insecure. Things like sharing accounts, using administrator accounts by the application, exporting sensitive data out of the database, and more.
Identify suspicious activity: An occasional human review of database activity can flag suspicious activity that has gone undetected by other means.
Identify gaps: Identify activity or possible activity not controlled by your current measures and with the potential to compromise the data.

Visibility is a key feature of a database security solution. In Core Audit, we call it Proactive Forensics. But regardless of the name, it should allow you to answer questions like who connects to the database, using what programs, from what machines/IPs, and what they are doing inside. You should have answers to questions like who accesses your sensitive information, when, from which programs, and using what SQLs. But these are just examples. The questions depend on the database activity profile, and that is what you need to understand to be able to control.

Aspects of Database Security

Database security solutions, built-in database capabilities, database auditing, IDS, IPS, and other needed capabilities are all intertwined. So, let us take a look at database security to understand what is important, what we need, and how we secure a database.

Basic Security

Basic database security is built into the database and involves authentication (users, passwords, etc.) and authorization (privileges, permissions, etc.). It may also include network encryption (data in transit) and file or disk encryption (data at rest). Additionally, there may be configuration parameters related to security that need to be controlled.

All of these elements require proper setup and periodic review. A change control process is crucial for maintaining control over basic security settings, and a database security solution should allow you to monitor those.

Change Control

Change control is based on the principle that controlling changes to the database (including basic security) ensures an ongoing secured environment. Database change-control aims to control configuration, objects, users, privileges, and permissions.

You exercise control through two mechanisms: an approval process for change requests before performing changes and an auditing process identifying all the changes to verify their approval. You can implement the audit process through metadata snapshots or using activity control. These should be part of a database security solution.

Vulnerabilities and Patching

Vulnerabilities play a minor role in database security. Most databases are mature systems with few undiscovered vulnerabilities. While vendors release regular patches, those usually address vulnerabilities that are difficult to exploit and rarely remote. Nearly all breaches occur through means other than unpatched or zero-day vulnerabilities.

Vulnerability scanning is not usually part of database security solutions but part of solutions that scan for vulnerabilities in many types of systems, not only databases.

Compliance

Compliance refers to adherence to laws and regulations designed to protect specific data, including personal information, credit cards, financial data, healthcare and medical information, and more. To prevent unauthorized disclosure or manipulation, the law (and derived regulations) requires companies to protect this type of information.

Ultimately, compliance regulations require implementing a security strategy. Beyond the focus on specific types of information, compliance requirements differ from regular security in that they demand a formal structure with documentation. The documentation serves as evidence of the actions taken by the organization to secure the data, and auditors can review it. Compliance typically encompasses all other types of security but emphasizes adherence to a process and record-keeping.

Since compliance is, essentially, security, that is something most database security solutions provide. Compliance is not a feature of a security solution, but certain features are important for compliance. For example, reporting, tracking changes to policies and reports, a tamper-proof repository (a repository where records cannot be updated or deleted), a long retention period with a minimal storage footprint, etc.

Activity Control & Database Auditing

Database activity control is primarily about auditing (monitoring) but can also include advanced prevention (IPS). Activity control and auditing are crucial to database security because most attacks use legitimate accounts. As the primary control on real accounts, activity control is the core defense against most data breaches. For example, privilege abuse, stolen credentials, and SQL injection are all examples of attacks that only activity control can address.

Activity control includes four primary measures:

Visibility – Understand the database activity profile using proactive and reactive forensics. That is an important control in its own right but is also essential for designing the rest of the controls.
Traditional Auditing – Set up policies, rules, and reports on subsets of the database activity that are high-risk and low-volume. It is also known as declarative auditing, and it is a central control in regulatory compliance.
Anomaly Analysis – Identify changes in the behavioral profiles of users, applications, etc. This is a critical control for high-volume subsets of the activity as well as for reducing your time investment.
Preventive (IPS) – Applying restrictions by blocking SQL activity. For example, preventing privileged users from accessing sensitive data, enforcing separation of duties, and more.

Data Change Auditing

Some compliance regulations mandate record-keeping of all changes to specific types of sensitive data. That goes beyond tracking who made the change, when, and how? and extends to recording the exact values that changed. It is sometimes called before and after images. For example, when an SQL statement changes a salary from 1,000 to 2,000, data change auditing records both the original value or the “before image” (1,000) and the new value or the “after image” (2,000).

This type of auditing may require significant storage depending on the data you need to audit, how often it changes, and the retention period. There are different ways to implement this requirement, including application code changes, database triggers, redo logs, and advanced auditing technologies. Each method comes with a different cost, complexity, and performance overhead you should consider.

While auditing data changes is something that can be done using application code or database features, using capabilities included in a database security solution is easier, more comprehensive, and has a lower overhead.

Technologies

Database Capabilities: All databases contain some form of auditing functionality. The benefit is that it is free. The downsides are lack of reporting, long-term retention, high overhead, and limited capture capabilities. By itself, this is an enabling feature, not a solution.
Tools using Native Auditing: There are inexpensive solutions based on built-in database capabilities. Their only value is offering the reporting and retention functionality missing in the built-in database capabilities. However, they suffer from the same high overhead and limited capture limitations. They can be acceptable for simple compliance needs but offer almost no security or protection against realistic threats.
Network-based Solutions: These high-end tools infer database activity from network traffic capture. They usually use local kernel drivers to capture local activity, but their capture is incomplete, so they are still easy to bypass. Local capture also creates a high network load when transmitting the local traffic to their server. Their primary focus is usually compliance, so they tend to have limited visibility and analysis capabilities. In other words, they have limited capabilities against realistic threats (abuse of privilege, SQL injection, and more).
Agent-based Solutions: High-end solutions like Core Audit connect directly to the database engine and see everything that happens in the database with low overhead (encrypted connections, local activity, and internal database activity). Core Audit also includes in-depth visibility, powerful repositories, and strong analysis capabilities, resulting in superior security against any threat.
Performance, APM, and other Solutions: These are solutions meant for performance and other purposes that are, sometimes, repurposed for security. They are inadequate because they are designed to focus on activities that generate load on the database instead of on activities that pose a security risk. Malicious SQLs usually create almost no load on the database, are rarely captured by the underlying technology, and, even when captured, will likely be ignored by the solutions.

Best Practices for Database Auditing and IDS

These best practices for database IDS and auditing solutions will help you achieve a strong security posture. It is important to note that while Core Audit can easily accomplish all these recommendations, other solutions might lack some of the capabilities.

Categorize Risks by Account Type: Databases have different types of accounts like DBAs, application accounts, analyst accounts, accounts for tools, and more. Each type of account has a different activity profile and is subject to different attack vectors. That means that to get effective security, each type of account will require different reports, anomalies, preventive measures, etc.
Start with Proactive Forensics: Before designing controls, use Proactive Forensics to understand your database activity profile and the activity profile of each type of account.
Implement Overlapping and Compensating Controls: Whenever possible, utilize several controls of different types to take advantage of their strengths and mitigate their weaknesses. For example, compliance reports provide a different kind of security than anomaly analysis, which differs from occasional forensic reviews. Using all three types against a threat will result in better protection.
Secure DBA and Privileged User Accounts: These accounts present a significant threat due to their unlimited power. They are subject to internal threats (abuse of privileges) and external threats (stolen credentials). Implement session controls to detect compromised accounts and activity controls to monitor DDLs, DMLs, sensitive data access, and time of day activity. Consider preventive measures (IPS) to limit DBA access to the data schema and to enforce separation of duties.
Secure Application Accounts: Application accounts have access to all data and are difficult to monitor due to high activity volume. Utilize session controls to detect unauthorized usage and activity controls like anomaly analysis to identify suspicious activity. Consider change control monitoring to validate change approval and preventive measures to limit activity sources.
Address Local Access Risks: Mitigate risks associated with local access to the database server. That is a popular attack vector in data theft and ransomware attacks. Implement session controls to monitor local connections and activity controls to identify suspicious activities.
Control Other Accounts: Identify and assess risks posed by accounts used by analysts, managers, and more. Implement session controls, activity controls, and others depending on the account permissions and activity profile.
Mitigate New, Unapproved, and Dormant Accounts: Detect and control the creation and usage of new accounts, changes to privileges, and, generally, activity from previously unseen accounts (like dormant accounts). Control is two-pronged: First, using session controls to monitor connections from infrequent accounts and any activity they perform. Second, reporting on DDLs related to accounts and privileges.
Secure Copies and Data Extractions: It is difficult, if not impossible, to secure data once it leaves the database. Manage risks associated with sensitive data copied out of the database. Production operations like replicas and backups require equivalent defenses like all sensitive databases. Non-production copies, like data for testing and development purposes, require data masking. However, data extraction for external analysis in Excel and other tools is a concern. If possible, try to avoid these. If not, the data should be masked or secured in another way. Implement activity controls to identify high-volume data access and an occasional forensic review to identify risky practices.
Improve Control Effectiveness: Perform a gap analysis on the effectiveness of the controls. That includes estimating false negatives (undetected events) and unknown attack potential (like zero-day). There are three methods for gap analysis: (a) based on events flagged by one security measure and not another (see compensating controls). (b) use activity profile analysis in proactive forensics to identify potential control weaknesses. (c) with a red team (or DBA team) that attempts to penetrate the defenses.

These practices will ensure a strong security posture. But, keep in mind that organizations usually follow a maturity path, and taking an incremental approach is often more realistic. For more details, contact us to get the complete Controls Guide. The guide includes a lot more detail and recommendations.