Many customers need to define requirements but building a good requirement document for database auditing is not simple. The reason is that customers who are new to the world of database auditing you are unlikely to be aware of the pitfalls and limitations products have and produce requirement documents that don’t protect them from such limitations.
To overcome this, many customers choose to copy in whole or in part requirements from various sources. This results in documents that are overly tailored to particular products and that require things that the customer doesn’t need or want.
The Requirement Wizard
This Requirement Wizard will allow you to choose the requirements that are relevant to you and explain when and why each requirement is important. It is also completely anonymous and does not send any information to Blue Core Research. When you’re finished, you can save or copy the result but Blue Core Research will have no knowledge of your choices or the resulting requirements.
Expand each item below and select the requirements that are relevant to you. As you select items the document will be created at the bottom. When finished, press the print button at the bottom and it will allow you to save the document you created.
Activity Sources
It might seem silly to specify every type of activity source you need to audit, but many technologies are limited in the type of activities they can audit. You must, therefore, specify exactly what type of activity you would need the auditing solution to audit.
Local Activity
Some solutions have limitations on capturing local database activity. That means that if a DBA or an attacker is logged into the database machine, the solution will not be able to see their activity in the database. Some solutions have different performance or other limitations when capturing local activity. If local activity is important, you should specify it
The solution must be able to capture local database activity. Any connection that originates from the database machine and connects locally to the database must be audited. All the requirements in this RFP, including performance requirements, must be satisfied when local database activity is captured.
DBA & Privileged Activity
Some solutions have limitations when monitoring privileged activity. Sometimes, privileged activity is not audited for certain privileged users. Other times privileged users can prevent their activity from being audited or delete it from the audit trail afterward. In some implementations DBAs deploy the auditing rules used to audit them. If privileged activity is important, you should specify it.
The solution must capture activity from all DBA and privileged accounts including built-in database accounts. DBAs must not be a necessary part in the deployment of auditing rules and policies. Privileged users must not be able to prevent their activity from being audited and must not be able to delete or alter their activity records.
Encrypted Activity
Some solutions don’t support auditing encrypted database connections. Some require additional components to support network encryption which can come with additional complexity, installation, and cost. If you are currently using network encryption in your databases or are planning on doing so in the future, you should specify it. Also, consider that some databases always allow encrypted connections and if you wish to audit those connections you would need network encryption support.
The solution must be able to audit encrypted database connections. All the required functionality must be available when the database network traffic is encrypted.
Internal Database Activity
Users can generate activity inside the database. Such activity can be generated by stored procedures, triggers, or anonymous blocks. It is particularly important when auditing DBAs since they always have the ability to generate such activity. It is also important in SQL Server where it’s common practice for applications to use stored procedures. Many solutions cannot capture internal database activity.
The solution must be able to capture internal database activities such as activity generated inside stored procedures, triggers, and anonymous blocks. The solution must be able to capture SQLs generated dynamically (e.g. using “execute immediate” in Oracle or “exec(‘sql’)” in SQL Server).
Batch / Block Activity
In many databases, it is possible to submit several SQLs at once. For example, In Oracle, it can be done with a begin/end block, and in SQL Server it is called a batch and is the normal way to submit SQLs. Identifying individual SQLs in a batch is particularly important when auditing DBAs since they often create such batches. If individual SQLs cannot be identified, it can become difficult or impossible to write auditing rules for it. This is particularly important in SQL Server because T-SQL does not require separators between consecutive SQLs in a batch.
The solution must be able to identify and audit individual SQLs in blocks and batches.
Short Running SQLs
Some solutions have difficulty capturing SQLs that execute quickly. However, in security auditing, fast running SQLs are often the most problematic. Long-running SQLs tend to be from reports or heavy application activity and are more important for performance tuning, not security. If you consider short running SQLs important, you should specify it.
The solution must be able to audit all SQLs regardless of how quickly they execute or in what order.
Database Overhead
One of the biggest concerns in database auditing is the impact on database performance. Depending on the technology that impact may affect different resources.
Coverage
The Core Audit Full Capture technology always captures all the database activity and does so with extremely low overhead. However, some technologies with higher overheads can reduce that overhead by limiting the activity being captured. It is, therefore, important to define what is expected to be captured under the overhead guidelines you specify. Please edit this section to reflect your needs.
All the requirements in this document, and specifically the overhead requirements are for auditing [all] the database activity in a database that executes approximately [10,000] SQLs per second and uses a network bandwidth of approximately [1 gbit].
CPU
CPU overhead is the most well-known overhead and it is important to define limits. Some solutions might hide their overhead in kernel-drivers.
When auditing the activity defined above, the solution will not create an overhead of more than 5% of the CPU power available on the database machine. This overhead requirement is for all software components of the solution including kernel drivers.
Network
Database auditing solutions create massive network overhead to the point of requiring an additional network card to be installed on the database machine. If such requirements are problematic in your environment, you should specify limits.
When auditing the activity defined above, the solution will not consume more than 5% of the database network bandwidth.
Latency
Some database auditing solutions create latency. Latency is a delay in communication to the database machine and back. Even a small latency can add up and significantly slow down chatty applications that submit a large number of short SQLs. This is especially common in some OLTP systems.
Reverse proxy solutions are an example of solutions that introduce latency. Some solutions introduce latency only when operating in certain modes. It is difficult to define limits on latency because its effect depends on the chattiness of your application. If you consider latency to be a problem you should require no latency.
When auditing all the activity sources and providing all the functionality required by this document, the solution should not cause any latency in database communication.
Out-of-the-box Value
An important feature for many customers is the out-of-the-box value. In other words, how much effort does it take to meet your basic needs? The question goes back to defining the basic needs, but below is a shortlist of basic needs that most customers require. Please add additional requirements based on your needs. Core Audit uses wizards to help achieve out-of-the-box value, and these requirements are from the 4 most commonly used wizards.
Sessions
Session auditing is the auditing of successful and failed logins. It is extremely common to require such reports.
The solution will allow us to quickly set up policies and reports about successful and failed logins to the database.
DBAs & Privileged Users
Auditing activity from DBAs & Privileged users is one of the most common requirements. Since DBAs rarely execute DMLs, it is helpful if DML activity can be separated out.
The solution will allow us to quickly set up policies and reports on DBA & Privileged account activity. The reports should include a separate report for DML activity.
Sensitive Table Access
Auditing access to sensitive tables is a very common requirements.
The solution will allow us to quickly set up policies and reports that monitor access to sensitive tables.
DDLs & DCLs
A very common requirement is to audit DDLs (changes to the meta-data of the database) such are creating or altering the table structure. In some terminology. the term DCL is used to separate out a category of DDLs that deals with the authentication and authorization system.
The solution will allow us to quickly set up policies and reports that monitor DDLs as well as special reports on DCLs (DDLs that deal with authentication and authorization).
Reports, Alerts & Integration
Reporting and alerting is the most important functionality to many customers. As a result, it is the most well supported functionality in most products. It is important that specify what you need to accomplish, but it is highly likely that most solutions will be able to achieve it.
Out-of-the-box
A common requirement is for out-of-the-box reports and alerts that can be easily customized to the monitored environment.
The solution will include out-of-the-box reports and alerts that can be easily customized to the monitored environment.
Easy to customize
Every database is different which means that reports and alerts always need to be customized to the monitored environment. Additionally, each compliance officer or auditor can require different information to comply with the regulations. Over time both the environments and the requirements change causing the need for additional adjustments. To avoid a significant time and cost investment you can require reports and alerts to be easily customized.
Reports and Alerts must be easy to customize to meet our environment and compliance requirements. Customization must include control over both the data being displayed and the filters on the data. Customization should be possible both during the initial setup and as needed on an on-going basis.
Reports
Compliance often requires reports on various aspects of the activity in the monitored environment. The reports serve as a paper trail demonstrating that the environment has been monitored on a regular basis. If you need database auditing for compliance purposes, you would need reports that are automatically generated.
The solution must be able to automatically produce reports on all aspects of the monitored environment that have been previously detailed. The solution must have a built-in capability to automatically schedule these reports and all components needed for this automatic generation must be included.
Alerts
A common requirement is for solutions to send alerts. Alerts serve two main purposes: (a) to be notified as soon as something happens without waiting for the next report and (b) to monitor a large number of aspects without getting a large number of daily reports. Alerts can achieve this when the filters are turned so that they normally do not produce output, but require the alert to be detailed and include all the offending information.
The solution must be able to produce alerts on all aspects of the monitored environment that have been previously detailed. The solution must have a built-in capability to automatically produce these alerts and all components needed for this must be included. The alerts must be detailed and include full details of the offensive activity.
Email Delivery
Many customers require reports and alerts to be deliverable by email.
The solution must be able to send all the reports and alerts via email.
Report File Formats
Some customers need reports to be provided in particular file formats such as html, PDF, and csv. If you have such requirements, specify the relevant file formats.
The solution must be able to produce reports in html, PDF, and csv file formats.
Report management
Some customers wish the auditing solutions to include report management capabilities. Report management capabilities allow users of the solution to review reports and alerts, classify them (e.g. reviewed, require further action, etc), and retain them in an organized manner. This is not an alternative to a proper ticket management system, but simply a way to handle the volume of reports.
The solution must include a report management system that can classify reports and alerts into various custom categories and retain them for long periods of time.
SIEM & Syslog
Some customers wish to integrate database auditing into their SIEM solution. SIEM normally accepts information via the Syslog protocol. While no SIEM can handle the load of all database auditing activity, they can handle forwarding of a small number of specific events.
The solution must be able to forward specific events directly to a SIEM system via the Syslog protocol. The solution must allow customization of the events forwarded to the SIEM system to include any event that can be reported or alerted on.
Custom Actions & Integration
Some customers require various other integrations, actions, or automatic reactions. Examples include sending SNMP traps, running scripts that can react to sever security threats, custom notifications, and more. If you have specific actions or integrations in mind, please specify those. The language included by this section fits if you generally wish to have the option of performing such integrations.
The solution must be able to able to run custom scripts in response to reports or alerts. The scripts must be given the report or alert information allowing them the freedom to integrate with any external system or react to the situation accordingly.
Forensic Investigations
Forensic capabilities in database auditing can serve two important and distinct purposes. The first and most obvious is reactive. That means in response to a breach, a suspicious event, a notification etc. The second which is just as important is proactive. That means in order to gain understanding of the activity, to plan the controls (policies & reports), to identify gaps in the controls, etc.
Session Forensics
Session forensic aims to inspect who logged into the database, when, and where from. It also aims to inspect failed login attempts. This is the most basic view of access requests to the database.
The solution must allow investigation of who logged into the database, when, and where from.
SQL Forensics – Declarative Auditing
Declarative auditing is when the solution records activity based on rules specified by the customer. For example, recording DDLs, privileged activity, certain IPs or programs etc. This activity will usually be reported on, but it is also important to be able to provide forensic capabilities.
The solution must allow investigation of all recorded activity in the database both on a per-session basis, and across sessions. The solution must also be able to associate each activity with the session that performed it and provide all the session information.
All Activity – Source Forensics
In addition to declarative auditing, Core Audit also contains aggregate information about all the sources of activity. This type of forensics is very useful to know where activity was executed from and when. If you consider this to be important you should ask for it.
The solution should allow investigation of where any activity in the databases executed from. This investigation should allow looking into all the accounts, programs, IPs etc that executed activity in the database. The investigation should allow to find out when activity was executed from each activity source or combination of sources (e.g. a combination of a user and program).
All Activity – SQL Forensics
Another powerful aggregation available in Core Audit in addition to declarative auditing is about all the activity in the database. This type of forensics is very useful to know what SQL construct was executed, who by, and when. If you consider this to be important you should ask for it.
The solution should allow investigation of all the SQL constructs that executed in the database, who by, and when. This investigation should allow looking into all the accounts, programs, and activity that executed in the database. The investigation should allow to find out what SQL construct executed, from which user and program, and when.
Graphical Forensic Investigation
Forensic investigations are often done with tables that contain vast amount of information. However, it is often useful to be able to perform graphical investigations using timeline graphs, pie charts, bar charts, etc. These are useful both to understand the data and to present it to others.
The solution should display at least some of the forensic information using graphs and charts that allow for a faster and easier understanding of the information and to quickly locate potential security problems.
Automation
Unlike declarative auditing that records whatever you setup policies for, automation aims to identify suspicious activity and point it out to you. There are many technologies that can be used for automation, so it’s difficult to write product agnostic requirements. It is, however, possible to specify certain types of unusual activity that you would expect the automation to identify, keeping in mind that it will likely be able to do much more.
Detect New Accounts
A simple expectation is to be able to identify when a new account connects to the database. Sometimes accounts are newly created and sometimes old and dormant. However, when you see activity from an account that hasn’t recently connected to the database, it is reasonable that the automation would alert you of the situation.
The solution should contain an automation that will alert the customer when there is activity from an account that hasn’t been recently active.
Detect New Machines/ IPs
A similar expectation to New Accounts is to be alerted when there is activity from a machine or IP address that hasn’t been recently active.
The solution should contain an automation that will alert the customer when there is activity from a machine or IP address that hasn’t been recently active.
Detect New Programs
Another dimension that it is important to identify new activity from is programs. You should be alerted when there is activity from a program that hasn’t been recently active.
The solution should contain an automation that will alert the customer when there is activity from a program that hasn’t been recently active.
Detect New Connection Source
A marginally more complex requirement is to identify new activity from a combination of activity sources. For example, a user that has been recently active and a program that has been recently active, but that user hasn’t recently used that program.
The solution should contain an automation that will alert the customer when there is activity from a combination of users, programs, and machines / IPs that hasn’t been recently active. For example, is a user uses a program they haven’t recently used or connects from a machine or IP address they haven’t recently connected from.
Detect Potential SQL Injections
A more complicated requirement is to identify potential SQL injection. In Core Audit this can be generalized into any suspicious SQL construct, but at the very least the solution should have some technology that will alert of a potential SQL injection.
The solution should contain an automation that will alert the customer when there is a potential SQL injection from the application.
Detect Activity Time Anomaly
Another type of activity that should raise an eyebrow is activity at an unusual time of day. For example, a user that normally works 9-5 running something at midnight.
The solution should contain an automation that will alert the customer when a user, program, ,machine / IP, or combination of those is active at an unusual time of day relative to their recent activity profile.
Detect Activity Volume Anomaly
Anomalous behavior can also be in volume of activity or data. This particularly important in order to detect extraction of unusual volumes of data.
The solution should contain an automation that will alert the customer when a user, program, ,machine / IP, or combination of those is processing or extracting an unusual amount of data.
Discovery
A subject that is not part of database auditing but often available in database auditing tools relates to environmental scanning and discovery. These type of capabilities tend to support a customer using the database auditing solution.
Network Scaning
Some customers wish to scan their network for databases. These scans can help with database inventory as well as to identify new databases that are brought online. If you consider this to be important, specify it.
The solution should be able to scan the network and identify databases running on particular port ranges. The solution should also allow identification of new databases identified by these scans.
Sensitive Data Discovery
Some customers wish to scan their databases for sensitive data. Scanning is done by sampling some rows from every column and performing pattern matching to see if the data looks like what the customer is searching for.
The solution should be able to scan the database for sensitive information by identifying data patterns that the customer is searching for.
Permission Discovery
A common requirement is to identify current accounts, privileges and permissions as well as changes those.
The solution should be able to identify current accounts, privileges, and permissions and report or alert on changes to those.
DBAs & Privileged Account Discovery
A common requirement is to identify DBAs and privileged account as well as other accounts with elevated privileges.
The solution should be able to identify DBAs and privileged accounts as well as accounts with elevated privileges. The solution should also report or alert on changes to those.
Preventive Measures – SQL Blocking
SQL Blocking allows extending preventive measures well beyond built-in database functionality. These include practical requirements like reducing the risk of DBAs & Privileged accounts, limiting access to sensitive data and more. While powerful, special care must be taken when deploying such measures and it is usually best done as part of a maturity process.
Not Currently Required
Properly implementing SQL blocking requires a solid understanding of the database activity and we don’t generally recommend first-time buyers to jump into it. However, if SQL Blocking is something you are considering in the future, you should specify future requirements so that you don’t lock yourself into a product that will not be able to provide you with these capabilities in the future.
SQL Blocking is not currently required from the solution. However, the solution must be capable of providing the capabilities specified in this section if and when we choose to acquire the necessary additional modules.
Block Activity
The solution is expected to be able to block activity based on customer defined rules.
The solution must be able to block SQL activity based on customer-defined rules. The activity must be blocked and not executed by the database. For example, a query will not return a result set, and a DML will not make modifications to the tables.
Latency
When enabling SQL Blocking, many solutions change the way they deploy or deploy additional modules in a way that will introduce significant latency in some or all of the database connections.
Specifically, some solutions in some operating modes will send messages to their central server for every packet received by the database and wait for a response before letting the packet through. This makes database connections extremely slow.
All the requirements in this document must be provided when SQL Blocking is used, including the overhead requirements. When using SQL Blocking there should be no additional latency on any session including sessions that are being monitored for blocking.
All Activity Sources
Since SQL Blocking is sometimes provided by separate modules that are deployed differently, it is important to reiterate the need to cover all the activity sources previously mentioned.
Specifically, some solution will only allow blocking of network activity. This is a significant limitation especially when DBAs & Privileged accounts need to be restricted. Also, some solutions will offer multiple blocking options allowing you to choose between blocking only network activity and additional latency.
The solution must be able to block all activity sources specified in this document including local activity and encrypted activity. The solution must be able to block all these sources without adding additional latency and without allowing the activity to execute in the database.
Internal Database Activity
Most solutions will not be able to block activity that occurs inside the database such as dynamically generated SQLs. This is a significant limitation if DBAs & Privileged accounts need to be restricted.
The solution must be able to block internal database activity such as activity generated inside stored procedures, and anonymous blocks. Specifically, the solution must be able to block SQLs generated dynamically (e.g. using “execute immediate” in Oracle or “exec(‘sql’)” in SQL Server).
DBAs
Some solutions require DBA involvement in the deployment of blocking rules. If you plan on using SQL Blocking to restrict access to privileged accounts, it is important that DBAs are not an essential part of the deployment of blocking rules.
The solution must not require DBA or privileged users to be involved in the deployment of SQL blocking rules and policies.
Best Practices & Log-Only Mode
SQL Blocking has the potential to disrupt production activity for various reasons including a mistake done by the customer in the definition of the blocking rules. One of the methods to reduce that risk is by deploying in log-only mode prior to enabling the blocking. It is also expected that vendors provide best practices for reducing the various risks in deploying SQL Blocking.
The vendor must provide best-practices for reducing the risk of deploying SQL Blocking. These best practices must cover various risks include mistakes done by the customer in the definition of the blocking rules. The product must also support log-only mode in which some or all of the blocking rules will log when activity should be blocked but will not actually block it.
Limit Privileged & DBA Account Access
One of the reasons for deploying SQL Blocking is to limit privileged and DBA accounts. Specifically, to prevent such accounts for accessing the data schema. If this is one of the expectations you have from SQL Blocking, you should specify it.
The solution must be able to restrict DBAs & Privileged Accounts. Specifically, the solution must be able to prevent such accounts from accessing the data schema.
Protect Sensitive Tables
One of the reasons for deploying SQL Blocking is to limit access to sensitive tables. For example to ensure that certain tables can only be accessed by the application account and from the application servers. If this is one of the expectations you have from SQL Blocking, you should specify it.
The solution must be able to restrict access to sensitive tables so that they could only be accessed by the application account and from the application server. Any access no from the application account or not from the application server must be blocked.
Provide Separation of Duties
SQL Blocking can create separation-of-duties in the database. This is done by blocking risky activity and giving security personnel the ability to grant particular users a temporary bypass from these restrictions. If this is one of the expectations you have from SQL Blocking, you should specify it.
The solution must be able to create separation-of-duties in the database. The separation of duties will be done by privileged personnel requiring temporary permission from security personnel in order to perform certain actions.
Day and Time Restrictions
SQL Blocking can restrict activity only on particular days or times. For example, to restrict activity at night and on weekends. If this is one of the expectations you have from SQL Blocking, you should specify it.
The solution must be able to block activity only during certain days and times. Specifically, the solution must be able to block activity only at night and during weekends.
Activity Source Restriction
One of the benefits of SQL Blocking is to block by activity-source such as IP address or subnet. If this is one of the expectations you have from SQL Blocking, you should specify it.
The solution must be able to block activity by activity-source such as an IP address or subnet.
Customization
SQL Blocking must always be customized to fit each environment. A false-positives means that production activity was blocked, so it is important to be able to quickly and easily customize every aspect of the blocking rules to fit a specific database.
The solution must provide a simple interface to quickly and easily customize all blocking rules, including the ability to immediately put them into effect.
Audit Server Hardware
Depending on the solution, the Audit Server hardware requirements can be significant. These requirements depend on various parameters such as the amount of activity that needs to be processed and stored. These are some suggestions on how to specify these requirements.
Software / Hardware Option
Some solutions are offered as appliances while others are offered as software. Software solutions are usually cheaper and allow the customer to deploy on a virtual machine. If it is important for you that the solution can be deployed on a virtual machine without installing additional hardware in your data center, you should specify it.
The solution must be provided as a software solution that can be deployed on a virtual machine.
Customer-Provided Hardware
Some solutions have extensive hardware requirements such as high CPU requirements or a large number of high-performance network cards. It is recommended that you specify what reasonable hardware you can provide and require any additional hardware to be provided by the vendor.
The customer can provide a virtual machine with [1] network card, up to [4] CPU cores, and [8] GB of memory. If the solution has a higher hardware requirement, the vendor will need to provide that hardware.
Storage
Storage requirements can be significant in many solutions and depend in large part on the amount of data you plan on storing. We recommend that you specify what reasonable amount of data you expect to capture and what reasonable storage you can allocate for that. The vendor to provide any additional storage.
We expect to store up to [500,000] SQLs per database per day and need to retain this activity online for [10] years. We can provide [1TB] of storage for this data. The vendor needs to provide any additional storage.
