A Guide to Peak Software Performance for Magnet AXIOM
In this installment of our series, we will cover key software settings within Magnet AXIOM and AXIOM Cyber that will ensure you are getting the most from your hardware investment—following up on part one in this series, in which we focused on hardware options to boost your case processing performance.
Additionally, we’ll cover some frequent issues that arise during case processing that can have a significant impact on the overall performance of your system.
Program Settings
The idea of dividing the workflow for case processing—including things like evidence file storage location, case file storage location, temporary file storage—was mentioned in part one relating to hardware.
To take advantage of multiple physical disks installed in your system, when you first install Magnet AXIOM or AXIOM Cyber, there are a few default settings you will likely want to change. This will help to best utilize your available physical disks and how they are used for certain workflows. Spend a few minutes and review the options in the settings menu of AXIOM Process to configure the proper selections for temporary file location used during case processing and hash database location.
The default locations for these may not provide the best performance on your system and consideration should be given to the type of use for these case processing functions—temporary files are both written to disk and read by AXIOM. With any of the available hash sets, we are primarily concerned with read speed during case processing whereas write speed is a secondary concern.
Fortunately, these are persistent settings, so they will only need to be configured once to best suit the hardware present in your workstation and these selections will remain each time we launch AXIOM Process.
Also, in AXIOM Process, when creating a new case be sure to confirm that you’re storing the case files and any acquired evidence on the appropriate volumes to not run into disk throughput issues.
The main CASE DETAILS screen in AXIOM Process makes it easy to designate a location for created case files and for acquired evidence, when using AXIOM Process to create forensic images of your evidence sources.
When working with previously acquired images, simply store those on an appropriate physical drive on your system and point AXIOM Process to those items when adding them via the select evidence source screen.
The overall goal here is configuring AXIOM to use separate physical volumes for case file storage and evidence file storage, eliminating any performance bottleneck when reading from an evidence file and trying to write to the designated case file location.
Another setting to mention: in AXIOM Examine, you can access the program settings via the Tools menu and then designate an appropriate location for temporary files created in AXIOM Examine from the PROCESSING tab.
While this will not have as significant an impact on performance as the temporary file location in AXIOM Process, the default location is within the user’s AppData folder on the system drive and choosing a different physical disk may provide performance benefits.
Anti-Virus Software
There’s no easy answer here, compliance with organizational policy or a regulatory body may present challenges.
But antivirus software can (will is probably a better descriptor) slow down your case processing. As items are extracted to temporary file locations during processing, including child objects being extracted from a container (like JPGs from a ZIP archive or email messages from a PST), those items will be scanned by your anti-virus software—unless you’ve created appropriate exceptions for the target location.
The standard answer of ‘it depends’ applies here. It depends on the specific A/V solution you are using, and it depends on the specific evidence you are processing and its contents. I have had cases take significantly longer to process when I have forgotten to disable A/V software or haven’t created appropriate exceptions when setting up a new system, and I have also had case processing fail altogether, more on antivirus and performance issues here.
Another factor is that at some point your evidence sources will likely have malware in them. If this has not happened to you, give it time. Depending on the type of investigations you are involved with, this may even be the very thing you are looking for and there is no one size fits all approach to this.
Anti-virus software developers—and Microsoft with their approach to system security—are doing a great job at trying to protect us all from the ever-present threat of malicious software. They’re doing such a great job that this software will often turn itself back on (or never fully disable) even when you have chosen otherwise.
For just a moment, remove the threat of malicious software from the scenario and just consider the likely behavior of endpoint A/V software that is still running on your workstation, without exceptions, created for things like the evidence storage location and temporary file location.
You may see that your computer gets a bit sluggish, as an example, say during case processing AXIOM extracts 10,000 email messages from whatever type of storage container they are recovered from within your evidence source. Without an exception to your evidence file location and temp folder location, your anti-virus software will happily start scanning away as this data is extracted and written to disk.
That can’t be good for performance, right?
This can result in undesirable behavior from your antivirus software. Maybe it just adds a [probably] significant amount of time to your overall case processing. Or, case processing slows to a crawl and then eventually crashes. Sometimes case processing will hang up and you are left wondering if the case is done processing or still going. Or, case processing slows to a crawl but finally completes and some items are not in your case as they should be because the antivirus software quarantined them before your forensic software could analyze them, so they are skipped (not an ideal situation).
None of those options seem very acceptable, considering the nature of this work. We need our cases processed sooner. We need access to relevant and actionable data now. We need to know that anything included in our original evidence source is also included in the case and was not secretly quarantined by our antivirus software.
If an evidence container holds malicious code that the antivirus software recognizes, it will work to quarantine that item to prevent your system from harm. This is usually a good thing, but the same anti-virus software does not usually send any type of report back to your forensic software notifying it of this action taken.
For the examiner, once that case has completed processing, they have no way of knowing certain items have been safely placed into quarantine by the antivirus software and are therefore may be excluded from the results.
Most anti-virus software gives the user an option to create an exception to exclude certain locations from being scanned, like your case file storage location and temporary folders. Additionally, there are plenty of commercial and open-source forensic tools that we all use which have components which may be flagged by anti-virus software, so exceptions for the installation location of these applications may be the easiest fix.
Dynamic App Finder
If you have used Magnet AXIOM or Magnet AXIOM Cyber, you may have been tempted by the option to “Allow AXIOM to find more artifacts”. If you have used this feature but are not quite sure what was actually going on in the background, please, right now, follow along with me and double-check something.
Launch your installation of AXIOM Process, select the Tools menu and then Manage Custom Artifacts. If you see tons of entries in there and have no idea where they all came from, keep reading.
Custom artifacts are great…unless these custom artifacts were created using the Dynamic App Finder and the process was not done correctly.
Occasionally, users think they are doing the right thing, only to later find they have improperly added custom artifacts into their installation of AXIOM or AXIOM Cyber.
If you aren’t familiar with it, more information on the Dynamic App Finder is available here. The Dynamic App Finder is a very powerful feature available in both AXIOM and AXIOM Cyber that can be used to provide artifact support for unsupported applications that use an SQLite database as their data storage mechanism by guiding examiners through the process of creating the necessary custom artifacts.
If this guided process is not performed correctly, then you are inadvertently creating a custom artifact that essentially has a bad set of instructions. That custom artifact will be used to search your evidence sources, using that same bad set of instructions, and well—garbage in, garbage out—you will get results in your cases that may not be useful. While custom artifacts can be an extremely powerful way to bring unsupported application data into AXIOM, we all know what Spider-Man said about great power…
During case processing, the Dynamic App Finder will typically locate many SQLite databases which are not associated with currently supported artifacts, with each database typically containing multiple tables.
Any tables contained within the discovered SQLite database files—which meet the criteria that the Dynamic App Finder is programed to find—are presented for review in the CREATE CUSTOM ARTIFACTS window that appears when case processing completes. Where things usually go wrong, is when a well-intentioned examiner enables the Dynamic App Finder during case processing and then chooses the ENABLE ALL button without first performing the manual process of previewing and mapping the columns from the various tables in those SQLite databases for relevant data.
Just think for a minute how an SQLite database for something like a basic chat application may work.
There could be tables for contacts, another for messages, and yet another for message attachments—each of these could potentially be surfaced by the Dynamic App Finder. If an examiner does not review the data contained within these three tables and make the appropriate decisions regarding the content of those tables, you can end up with a custom artifact for each table that adds a lot of noise to a case.
Where things go even further off the rails is when this happens repeatedly, within the same installation of AXIOM or AXIOM Cyber and you end up with dozens (I’ve even seen hundreds) of these not-so-custom custom artifacts. Then for every case that comes after this one and those custom artifacts are enabled, that is additional noise which gets created in the case, and additional time spent processing those data sources.
Custom artifacts are incredibly powerful, and you really should be using them. They can be written in XML or Python, downloaded from the Artifact Exchange on the Magnet Forensics website, or examiners can use semi-automated mechanisms such as the Magnet Custom Artifact Generator (free tool) or the Dynamic App Finder built right into AXIOM to create custom artifacts to use in their cases.
Just ensure that you are creating proper custom artifacts and are not just adding everything the Dynamic App Finder throws your way into a case, doing so adds a lot of unnecessary time to your case processing.
Magnet.AI, Please FIND ALL EVIDENCE For Me
Large Language Models (LLM) such as Open AI’s ChatGPT and Google’s Bard have been the topic of many discussions and articles recently. With the idea of using an AI model to do the work of a forensic examiner making an appearance occasionally.
I think we can all agree we’re not there yet and the day an AI model can replace the decision making and data interpretation skills a human examiner brings to the table means we all probably need to find new work.
However, I am big fan of using technology to assist human examiners—I need all the help I can get!
If you have used AXIOM or AXIOM Cyber for any length of time, you have likely seen the available options for Magnet.AI. Perhaps a great way to refer to these features are calling them assistive technologies.
Each Magnet.AI option is utilizing a model to search for and match characteristics—either of images or within the content of chat message threads. The features offered by Magnet.AI are not perfect but they can sure do a lot of heavy lifting for you with respect to the work you may be required to perform related to media identification and categorization and identification of explicit chat messages.
Just a quick overview, Magnet.AI leverages dedicated graphics processing units (GPUs) when they are installed in your workstation, and this can significantly decrease the processing time it takes to perform this work. Magnet.AI currently supports the use of video cards from the following NVIDIA lines:
- NVIDIA Tesla
- NVIDIA Quadro
- NVIDIA NVS
- NVIIDIA GeForce
- NVIDIA TEGRA
A supported NVIDIA GPU can provide significant speed improvements for several of the common tasks associated with Magnet.AI for image classification and finding visually similar pictures. This workload is mathematically intensive and because this is where GPUs excel, the speed enhancements can be dramatic.
As an example, on a sample data set with roughly 140,000 images, using Magnet.AI restricted to only CPU processing for categorization and a selection of image classification categories took over one hour to complete. Using the same data on the same computer but installing an NVIDIA Quadro P1000 GPU reduced that to around half the time. That is a significant time savings with a very modest GPU! On a different system with an NVIDA RTX 3080Ti, using the same data set and processing options, that time was reduced to around ten minutes. In the real world, you will typically have well over the 140,000 images used for this example, so the time savings can really add up.
A couple of notes regarding the use of Magnet.AI for image categorization: be selective with your choices and make a conscious decision about when to perform this option. Like other options in AXIOM, using Magnet.AI to categorize images is optional. And it may not be something you use on every case. If you are working a CSAM investigation, do you need EVERY category enabled?
What about for a drug investigation? Homicide? You get the idea. The categories that exist here are suited for a variety of investigation types but selecting them all just to FIND ALL EVIDENCE is not an efficient practice.
The good news is that you can perform this option whenever it’s convenient. Yes, there are situations where you just know you’ll be using this feature and may enable it during the case creation workflow in AXIOM Process, selecting the image categories as appropriate for the investigation. There are other times where you may choose to wait and perform this after initial case processing, and this is something that is worth talking about.
Knowing this can be a time-consuming process and that it may not be necessary to perform on every investigation, maybe consider holding off on performing image categorization with Magnet.AI until after initial case processing and you’ve had a chance to look around a bit to get an idea of what you are looking at in a specific case. This is a great thing to kick off right before you leave for the day, letting the computer work while you aren’t there waiting on it and this lets you take advantage of computing time which would otherwise be unused.
Because there is no singular approach to digital investigation, AXIOM gives you options. And there are no doubt going to be times where you need to perform image categorization while you continue to work on other aspects of a case. When this situation arises, use the appropriate system resource allocation option from the Magnet.AI dialog window.
Selecting the option to Optimize for speed will consume available system resources and make using that computer for other work impractical whereas selecting the option to Optimize for multi-tasking will strike a balance that would let you continue working through the case in AXIOM Examine.
This same system resource allocation drop-down menu is available in the dialog window for using Magnet.AI for chat categorization, as well.
It’s All in the Artifact Details
Alongside other features available in both AXIOM and AXIOM Cyber, the options provided within the Artifact Details section of AXIOM Process can be used to streamline your workflow and get you to actionable data quicker. Using a combination of these features may be appropriate for certain investigations and these can also be useful when trying to determine if a particular data source is relevant to the case. Another use appropriate for some of the listed features includes complying with search authorization.
Artifact Selection
While you don’t always know what a set of evidence items may hold, you often have an idea of what it is you are looking for. Instead of being tempted by the SELECT ALL button for a particular evidence source, take a considered approach.
Some of the available artifacts are disabled by default—you simply may not need them in every case you work. Enabling them will not only increase the overall case processing time, but you’ll also end up with an even greater artifact count to work through.
A good example is the operating system artifact for NTFS timestamp mismatch. While this is a great option when looking for evidence of time manipulation on the system, you may not need that data point on every investigation.
Another consideration for those times when you enable all the available artifacts during case processing, even if you ignore the resulting artifacts entirely during analysis, their existence in the case may have an impact on performance because your computer is having to do more work each time you perform actions within AXIOM Examine. Additionally, some of the artifacts which are enabled by default may be related to items that you aren’t likely to find, do not have interest in for a particular case, or need to be deselected to comply with scope requirements of the search.
The ability to create artifact profiles within AXIOM Process and then reuse them on subsequent investigations can help with this and they are also useful when creating a subset of artifacts to use for specific purposes.
As an example, consider that many operating system artifacts on a Microsoft Windows system are sourced from various Windows Registry files. You could then select appropriate artifacts from the Operating System artifact category and save that profile for later use.
Processing a subset of those artifacts which are sourced from Registry files may only take a couple of minutes but can provide tremendous insight into things like program execution, file and folder interaction, and user account information on that system. This can be a great way to triage the system to determine if it warrants further analysis; and it can be beneficial when a target of interest in an investigation has access to multiple computers and you are trying to determine which of them may be most relevant.
Parsing and Carving Artifacts
Acquisition and analysis of data sources consisting of terabytes of data and hundreds of thousands of artifacts can strain available resources and leave examiners feeling overwhelmed when it is time to review that data. Another feature you can leverage is the option to perform only parsing of artifacts during case processing and then later carving those same evidence sources for artifacts as a post-processing option.
This may not always be an ideal approach, but parsing operations are typically much faster than carving those same data sources for artifacts. You may find an initial pass using the option to only parse your evidence items for artifacts is a useful way to streamline case processing and begin review of those artifacts more quickly, leaving your computer to work on carving those same items for artifacts when you leave for the day, again taking advantage of otherwise wasted computing time.
Date Range Filtering
Along with other features available in AXIOM and AXIOM Cyber, examiners can utilize a date range filter during case processing to limit items shown in the Artifact explorer of AXIOM Examine to a relevant time frame. This is also useful to comply with scope requirements specified within your search authorization if that is a factor in your investigation.
Hash Sets
If you aren’t using the National Software Reference Library (NSRL) hash sets from the National Institute of Standards and Technology (NIST), available here, do yourself a favor and take the time to implement them.
The initial time investment pays off with every case you work afterwards. The downloads available from NIST include modern and legacy personal computer applications, along with Android and iOS specific options.
In practice, the Magnet Forensics Hash Sets Manager, available here, makes it easy to add these hash sets to all installations of AXIOM or AXIOM Cyber that are in your organization. The Hash Sets Manager provides a single-point interface to deploy and later update hash sets which can be used to quickly identify known instances of malicious or contraband files, or for screening out standard operating system and program files. Hash sets can help to streamline a wide variety of investigations, ranging from intellectual property theft to CSAM investigations.
Post Processing Options are Optional
None of us want to miss artifacts in a case and maybe that leads to us selecting more options and enabling more features in our tools when we process data. Just be intentional with what you select and have reasonable expectations. If media items are not the focus of an investigation, then it may not make sense to enable all those Magnet.AI image categories during initial processing.
As you review the available artifacts, you may find reasons to conduct additional processing or other artifacts. Maybe this is just a suggestion to a more selective approach to case processing and leveraging the available post processing options as just that—post processing.
Performing these during what would otherwise be unused computing time—like when you leave for the day and the computer just sits and patiently waits for you to return. This a great way to get things done without you sitting there waiting for another progress bar.
Closing
If you’ve made it this far and you have the appropriate hardware configuration along with ensuring the software settings are correct in AXIOM, then we may have reached the absolute cutting edge of performance—with respect to your system, evidence sources, and processing options.
Having used a variety of digital forensic tools during the years spent working in this field, I always wanted them to be faster. All the way back to using older dual and quad-core CPU computers and even when using multi-socket systems with mega core count. I have also watched things like hard drive capacity grow exponentially and the volume of data presented in just about every case do the same. All our forensic tools—commercial and open-source—do so much more work for us now, and this is a good thing. That also means part of this comes down to the simple fact that it takes time for any application to chew through the billions (now often trillions) of bytes of data we’re feeding it. Then we move that data around, write billions of bytes of data to temp files and case files, and all the other things we ask our forensic software to do for us during case processing.
Truthfully, the result of all this may be that it is time to consider an updated approach to this massive volume of data. Automation and leveraging cloud resources—and their scalability—are likely the next logical step. One human can only do so much work, no matter how many computers you give them to use.
If you aren’t familiar with the Magnet Digital Investigative Suite, the automation and orchestration features offered by a solution such as Magnet AUTOMATE, combined with the collaborative processes supported by Magnet REVIEW, should be on your list of things to research. Organizations that are using these products are reporting overwhelmingly positive results—getting to actionable data faster, finding evidence that would otherwise be lost, and eliminating a months-long backlog. We may have reached the tipping point in the battle against case backlog and data volume where this is no longer a problem that can be solved by throwing more people at it—technology is the answer.
Hopefully the suggestions presented here, and in the accompanying post related to hardware configuration options help you get the most performance out of Magnet AXIOM or Magnet AXIOM Cyber.
As our case sizes continue to grow, we must find ways to scale our efforts through effective use of technology. Still, I believe we need to all utilize a bit of strategy in our examinations, otherwise the backlog will continue to grow.
Links
- The official system requirements and suggestions are available at: https://support.magnetforensics.com/s/article/System-requirements-Magnet-AXIOM
- Using an NVIDIA GPU with Magnet AXIOM or AXIOM Cyber: https://support.magnetforensics.com/s/article/System-recommendations-for-using-Magnet-AI-with-a-NVIDIA-GPU
- AXIOM version performance testing: https://www.magnetforensics.com/blog/comparing-magnet-axiom-performance-speeds-through-the-ages/
- Antivirus Software: https://support.magnetforensics.com/s/article/Performance-issues-when-searching-or-processing-evidence