Not just the realm of TIPs. Automated collection of deeper-dive online research related to SOC investigations is an important tool in an analyst’s belt.
Ongoing data collection is a big part of an analyst’s job. Threat intelligence platforms provide a near-continuous stream of incoming data that sometimes warrant deeper investigation and online research. Analysts usually have a list of go-to sites for an ongoing investigation or a class of information. Regular checks on message boards, blogs and forums help keep tabs on changing content and monitor for new information and clues.
While these repetitive collections are essential, they tend to be time consuming and inefficient.
Automated collection tools can speed up the process, but some may introduce additional risks, exposing researchers to potential dangers: an adversary may notice patterns in analysts’ visits, match their digital fingerprints to specific online activities, or detect disparities in investigators’ locations and time zones relative to typical website traffic.
Security-conscious online researchers need automated collection tools that operate under the assumption that someone is always watching, and their missions may be compromised at any point if site owners suspect that they are being monitored.
Automated collection with tradecraft in mind
For online researchers, staying anonymous and blending in with regular website traffic is key to doing their job successfully and not exposing their organizations — and themselves — to potential dangers. And while certain tools, like virtual machines or VPNs, can help add some protection to automated collection tasks, they can’t provide complete anonymity, or shield networks from accidental exposure to malware.
Learn more about what VM and VPN services miss here >
Some organizations resort to maintaining “dirty” networks — dedicated computers that are disconnected from the organization’s main network and can help disguise the user’s affiliation. However, despite considerable expenses involved in setting up and maintaining these networks, they too fall short of good tradecraft. One lapse in management can leave the door open for adversaries to attribute activity to an individual or organization (e.g., unusual activity when analysts fail to mimic the “average” site visitor due to their language preferences, time zone settings, location, OS, etc.).
Specialized automated collection tools help mitigate risks
An automated collection solution that manages attribution and is accessible via a remote, cloud-based browser is the best way to ensure that researchers receive the needed protection and security while performing their collections efficiently and with precision. Here are a few essential capabilities that researchers should look for when selecting a tool to perform their automated collections:
Customize how you appear to sites and people you interact with online
Managed attribution allows a researcher to fully control their digital fingerprint, including language, time zone, keyboard and browser settings, location and more. For example, if you are collecting data from a site that receives most of its visitors from China, you will want to access it during local daytime hours — even if it’s the middle of the night in your location. Going in for collection at a regionally appropriate time will help you blend in with regular traffic (and get more sleep, too!).
What is managed attribution, and how does it improve online investigation? Read the blog here >
Additionally, you need to access sites from a regionally appropriate location. While VPNs can provide a spoofed IP addresses, some sites block VPN access altogether, knowing this is a common tactic used by investigators (not to mention the other security and tradecraft shortcomings of VPN for senstive research).
Leveraging a global research network that provides regionally appropriate points of presence (i.e., egress nodes) to avoid geoblocking or blocking due to VPN usage is another critical capability for automated — as well as manual — collection.
Schedule rolling collections at randomized intervals
Even with a completely customized identity, accessing the same sites at the same time daily or weekly can arouse suspicion. Rolling collections at randomized intervals with different digital fingerprints can give analysts the best chance at blending in.
Automate out-of-band translation
Collections may include sites in languages foreign to the analyst and require translation. Traditional translation requests of webpages can alert webmasters that a non-native speaker has visited the site, and that could arouse suspicion. An automated tool that performs translations out-of-band will can sidestep this issue, and help analysts appear as native speaker when collecting data from international sites.
Access via isolated browser and securely store collected data
An automated collection solution that uses a cloud-based browsing environment allows you to traverse both the surface and dark web with the same experience as any traditional browser but with added protections and security. With remote browsing, no web code ever touches the endpoint or network, eliminating the risk of malware infection.
In addition to securely accessing sites, collected information needs to be securely stored. Cloud-based storage off the corporate network is another key security consideration for automating collections. Stored data in the cloud should also be easily searchable and enable secure collaboration with fellow team members.
Repetitive collections are vital to SOC and CTI teams’ ability to respond to attacks and proactively identify and remedy weaknesses. Having the right tools at their disposal helps analysts keep tabs on suspicious sites, TTPs, leaked data and more — all without alerting adversaries, compromising their security or mission success. And scheduling collections for sites in other time zones helps reduce late-night shifts and frees up time during normal business hours to perform more strategic work.
How Silo can help
Silo for Research's automated collection extension is designed to help analysts increase productivity and enhance their tradecraft when performing collections. With its advanced capabilities (we’ve got you covered even for collecting videos), managed attribution and cloud-based isolated browser access, Silo for Research saves analysts’ time, while enhancing their security and privacy.
To learn more about how the Collector extension of Silo for Research automates online research collections, watch the demo now.
Tags Anonymous research SOC Threat intelligence
