Vantage Point: Forensic Readiness by Kristi Horton, InfraGardNCR Board of Directors

What’s the Problem? Why Bother?

As a forensic analyst, incident responder, and cyber threat intelligence professional, it is gut-wrenching when a client has been victimized and cannot seek justice for their losses. Most organizations are not adequately prepared to investigate what happened (or what is happening now), and lose the ability to press criminal charges, to sue in civil courts, to defend themselves from a lawsuit or external adversary, or to investigate insider threats. Even worse, some lack the records, logs or data to troubleshoot anomalies, or performance issues. They cannot determine if they have a technical problem or a malicious cyber incident. In civil litigation, the consequences of discovery violations can be steep. A judge can sanction a party or find for their opponent if data is not properly collected and preserved.

The cost of incident response, legal proceedings, fraud and theft investigations and insider threats are astronomical. Advance preparations can reduce those costs, accelerate investigations, and avoid penalties or losses.

How Do I Get Started?

Assuming the organization has a clearly defined and thoroughly communicated vision and mission and an established governance program, the first steps to preparing for a forensic investigation may come as a surprise, but they are the foundation for a competent security program and empower each organization to prioritize their efforts and focus on the most important activities and assets given its unique circumstances. Note these steps are part of a Risk Management Program, but also appear consistently in the Top 20 Critical Security controls. That is to say that these were deficient or non-existent in organizations that suffered large security incidents in recent history.

· Obtain and maintain an inventory of all assets: human, information, data, computing and network equipment, cloud-based facilities and services, intangibles like reputation and relationships, and physical assets as well.

· Prioritize the assets and activities that are the most critical to the organization’s vision, mission, and operations.

· Understand potential events (scenarios) that could affect those assets and activities most critical to the organization.

· Understand the potential motives, goals, and capabilities of the types of adversaries you might encounter.

The next steps to solving this problem is for each organization to consider its highest risk scenarios. Risk involves a measurement of the likelihood and impact of a given event. High risk scenarios might include:

· the theft of critical intellectual property such as research and development data, chemical formulas, recipes, manufacturing methods, or other trade secrets.

· The loss of a lawsuit (in which you are the plaintiff or the defendant) because relevant data was not properly preserved (or was not preserved at all).

· Major operational disruption from crypto-ransomware that prevents an organization from determining if the ransomware has been adequately remediated or how to prevent it in the future.

· Recurring intrusion incidents because the forensic data needed to determine the root cause or initial entry cannot be found or has been destroyed.

· Missing data or email messages cannot be recovered, and the cause cannot be determined because the logs needed to troubleshoot or investigate are unavailable. The date/time of the disappearance of the data cannot be determined.

· Inability to prove that an insider violated policy or even broke the law because log records and needed data are not collected or were not preserved.

· Inability to hold malicious actors accountable when they impersonate you to steal money from your customers or clients.

· Inability to detect hardware failures that are or will impact operations.

· Cannot find out how important data leaked outside the organization.

· Customer or vendor information is leaked or stolen resulting in privacy breaches, identity theft, and fraud losses.

· Inability to determine the extent of the damage of a security incident.

What should I keep?

The types of data and capabilities an organization requires during or after a given situation will differ depending on the organization, its IT infrastructure, its environment (geography, regulatory and legal), and the situation itself. Not all types of logs, records, or data are needed in every situation. Some logging enabled by operating systems, software, network appliances, or IoT/IIoT/ICS/SCADA devices is helpful and some logging that is helpful and needed may not be enabled by default. Organizations must take pro-active steps to identify potentially helpful records and data and plan to generate, collect, and preserve such data and information. As “logging everything” can become a very expensive proposition, organizations will not only need to determine what they might need, but how long they need to keep it. Packet captures, firewall logs, or netflows from legacy systems no longer in operations and that are over 5 years old, may not be useful or productive. Regulations or laws may dictate how long some records must be kept.

Some records and logs might fall into the following categories (note this is not a comprehensive list):

Personnel and Physical Security Records:

· Building and facility access records

· Surveillance video

· Travel authorization for employees

· Records for planned employee leave time

Volatile Data:

· Network traffic such as packet captures

· Random Access Memory (RAM)

· Actively running processes

· Internet Protocol (IP) tables and active connections

Network Records:

· DNS requests

· DHCP records

· Firewall Logs

· VPN Logs

· Incoming connection requests

· Outgoing Connection requests

· Netflow data

· DMARC reports for emails sent purporting to be from your email domain

· Logs of deep packet inspection results

Server Logs and Data:

· Email Audit logs including DMARC authentication failures for incoming messages

· Email header information (from, to, time, message-ID)

· Failed and successful logons including geolocation data if available

· Web Access Logs (web application servers)

· Application Logs

Host based and mobile device Logs and Data:

· File/Folder audit logs

· File system event logs

· Operating System Event logs

· Changes to configuration files/settings

· Host based firewall logs

· Login/logoff – both successful and unsuccessful

· Web browsing History

· Terminal/command line/PowerShell logs

· Application logs

· Changes to hardware configurations

· Use of external media

· Call logs

Logs and Records generated by Security software, tools, platforms or systems:

· Data loss prevention logs and records

· Antivirus scan logs and results

· Intrusion detection/prevention system logs

· Web gateway logs

Cloud based environments are no exception to the need for logs and records. The more infrastructure (and the more important the assets and infrastructure are) that an organization has “in the cloud”, the more critical it will be to establish a third-party risk management process and to establish and maintain strong lines of communication with cloud providers. When considering cloud-based products and services, questionnaires regarding security and confidentiality processes, procedures and controls must inquire explicitly about the logs and records available to organization administrators, how long the delays are before the logs are visible and available to the administrators, and any constraints on how long the logs will be preserved, or how much space is available to store the logs. Also, will administrators have to request logs from the cloud provider each time they wish to audit events? One may be shocked at how many cloud providers advertise their own security yet fail to provide even the basic level of audit logs to business customers. Cloud based providers also may be unaware of industry specific or regional compliance requirements that apply to your business or organization. Many parties to lawsuits have discovered that the efforts or features that some providers offer for discovery purposes fail to preserve the forensic audit trail. This is the result of an unfortunate disconnect between the rules of discovery and the rules of evidence/forensic procedure – it is not necessarily specific to cloud providers.

What else should I do?

Forensic Readiness extends beyond having the right configuration settings and log files. It also requires that there be people trained and educated in proper evidence collection, preservation, and chain of custody procedures. This can be an internal forensic response team or a third party provider. Employees and managers must all understand their role in the generation, collection, and preservation of data that may be needed in an investigation or legal proceedings.

Don’t forget that you may need to comply with privacy laws or that you may need to inform your employees about any network monitoring or data collection practices.

In the process of determining what could happen and thus what records may be needed, it will become apparent that modifications and additions need to be made to roles, policies, procedures, and playbooks.

That is a lot. Where do I get help?

While these considerations seem overwhelming, the experiences of security professionals, litigation support, and forensic professionals have been documented and there are resources to ease the burden for already taxed IT professionals. Do not feel pressured to achieve all aspects of readiness all at once. It is possible to get started with some basic data and to continuously improve the organization’s visibility into its own present and historical activity. Continuous improvements year over year will increase the capability the organization has to recover “deleted” data, to locate valuable records and information quickly, and to identify relevant evidence along with the ability to properly collect and preserve it.

Checklists are helpful, but there are benchmarks available to help configuration managers ensure that certain audit and logging features are enabled and configured to record events likely to be needed in forensic investigations. Microsoft offers customization templates for this purpose and provides a Security Configuration tool to help administrators audit configuration settings and compare Microsoft recommended baselines with other security configurations. More general benchmarks for a variety of operating systems, software applications, and platforms (including some cloud providers) can be found from the Center for Internet Security. Configuration settings in these templates and benchmarks are not limited to audit logging capabilities, but also to other settings as well. Some configuration settings will require additional tuning specific to the unique organization and environment. In addition to security benchmarks for configuration settings, many cyber insurance companies offer recommendations for security posture including configuration settings and recommended logs to collect and retain. Some providers offer some security assessment services. Others will offer reduced premiums for firms that meet certain baseline standards. As always, it is good to ask for help when you need it. Don’t forget to collaborate with your legal counsel and your local, state, and federal law enforcement partners. Consultation with experienced forensic, discovery, and incident response professionals can help you to consider scenarios you may encounter, and a variety of providers can help you develop tabletop exercises to help you learn the types of data you may need to collect and retain in case of a cyber incident. Be sure to ask your peers in information sharing groups and organizations which logs, records, and settings they found helpful (or missed when they needed it) based on their own experiences. InfragardNCR Sector Chiefs have established and information sharing platform and email listservs to facilitate coordination and communication among organizations.