Enumerating OS Artifacts: How Malware Profiles Sandboxes, VMs, and Targets

Malware rarely executes blindly. Before dropping payloads, it enumerates operating system artifacts to decide where it’s running and whether to proceed. This post walks through the main Windows targets -processes, files and directories, shared folders, registry keys, services, installed software, mutexes, named pipes, devices and drivers, usernames/hostnames, locale, and OS version- and explains how those checks reveal sandboxes and VMs or profile intended victims. We’ll also highlight how analysts can recognize these techniques and harden their labs to avoid false negatives during analysis.

Introduction

Modern malware doesn’t just drop and run -it first takes inventory of its environment. Enumeration is a reconnaissance step where the malware queries the operating system for signs that it’s in a sandbox, a VM, or the wrong target altogether.

The logic is straightforward:

1. Detect analysis environments. Look for artifacts such as VboxTray.exe, VMware services, or registry keys with VBOX__.
2. Validate the target. Check installed software, usernames, or organization-specific registry keys before running destructive payloads.
3. Ensure compatibility. Confirm the OS version matches the malware’s exploit or persistence method.

We’ll examine the Windows APIs (CreateToolhelp32Snapshot, RegEnumValue, NtQueryDirectoryObject), command-line tools (sc, wmic, reg query), and higher-level approaches (WMI, PowerShell) that malware leverages to extract these clues.

At the end, I’ll point to a side project where I’m implementing a lightweight enumeration tool in Rust that reproduces many of these checks in a safe, read-only way. This companion project is intended as a practical exercise, but the focus here is on understanding the techniques and recognizing them during malware analysis.

Process enumeration

One of the first things many malware samples do after execution is take a look at the processes running on the system. The logic behind this is simple: a real user’s machine usually runs applications like browsers, office suites, or messaging clients, but it’s much less likely to have tools such as Wireshark, Procmon, or VirtualBox guest services. By checking which processes are active, malware can decide whether it is in a genuine environment or in a sandbox built for analysis.

On Windows, this is usually implemented with the Tool Help API. The call to CreateToolhelp32Snapshot produces a snapshot of the running processes. Then, Process32First retrieves the first entry in that snapshot, and Process32Next iterates through the rest. With every iteration, the malware compares the process name against a set of suspicious targets. In pseudocode, this looks roughly like:

handle = CreateToolhelp32Snapshot(PROCESS_LIST);
process = Process32First(handle);
while (process) {
    if (process.name == "VboxTray.exe" || process.name == "procmon.exe") {
        TerminateProcess();
    }
    process = Process32Next(handle);
}

If a match is found, the program may simply exit, delay its behavior, or switch to a safer routine that avoids exposing its payload. Some families go further and try to conceal what they are looking for. Instead of embedding process names in cleartext, they store only their hashes. That means when an analyst inspects the binary, they won’t see "fiddler.exe" but something like 447c259d51c2d1cd320e71e6390b8495, which is the MD5 hash of that string. This adds friction to the analysis process: to understand what the malware is checking for, the analyst has to recognize the hash and resolve it back to the original process name.

Another variation is checking not the binary name but the window title of the application. This sidesteps a common evasion trick where analysts rename their tools to something innocuous, such as turning procmon.exe into notproc.exe. If the malware queries the title bar instead, it will still see “Process Monitor” in the window caption and identify the tool correctly. This makes simple renaming ineffective and forces analysts to either modify the binary of the tool itself or rely on stealthier setups.

From a defender’s point of view, spotting this behavior is important. A loop that repeatedly calls Process32Next and compares results should raise suspicion, because it means the malware is actively profiling its environment. For analysts, it’s also a reminder that a sample that “does nothing” might simply have recognized the sandbox and refused to reveal itself. I covered this topic in more detail, with examples and code snippets, in a dedicated article on process enumeration.

Some process names that commonly trigger these checks include:

• VMware: Vmtoolsd.exe, Vmwareuser.exe
• VirtualBox: VboxTray.exe, VBoxService.exe
• Analysis tools: procmon.exe, wireshark.exe, fiddler.exe, x64dbg.exe

Directories & Files

Processes aren’t the only clue that reveals a virtual environment. Malware often inspects the filesystem, looking for files and directories that shouldn’t exist on a real user machine but are common in virtualized setups or automated sandboxes. This technique is just as straightforward: if certain drivers, configuration files, or folder names are present, the sample concludes it is being analyzed.

A frequent target is the Windows drivers directory. For example, VMware installs drivers whose names start with vm, and VirtualBox creates files prefixed with VBox. By scanning the directory C:\Windows\System32\drivers\ for files that match these patterns, malware can quickly identify its environment. The code below shows a simplified routine that uses the API functions FindFirstFile and FindNextFile to iterate through files and look for a suspicious prefix:

handle = FindFirstFile("C:\\Windows\\System32\\drivers\\vm*");
while (handle != INVALID) {
    if (filename starts with "vm") {
        TerminateProcess();
    }
    handle = FindNextFile(handle);
}

The same logic can be applied to directories. VirtualBox installations typically create a folder under C:\Program Files\Oracle\VirtualBox Guest Additions\, while many sandboxes, such as Cuckoo, leave behind recognizable directories (C:\cuckoo\, C:\agent\) or configuration files (analysis.conf, analyzer.py). Stumbling across any of these artifacts is enough to alert malware that it is not in the wild but under observation.

What makes this particularly dangerous for analysts is that the technique is cheap, fast, and stealthy. File lookups are a normal operation for legitimate programs, so a few FindFirstFile calls don’t stand out unless you’re watching closely. Yet for malware, these checks act as an environmental fingerprint, ensuring payloads are only delivered to intended victims -or nowhere at all if the system looks suspicious.

Some of the most common filesystem artifacts checked by malware include:

• VMware files: C:\Windows\System32\drivers\vm*, C:\Windows\SysWOW64\vm*
• VMware directories: C:\Program Files\VMware\
• VirtualBox files: C:\Windows\System32\VBox*, C:\Windows\System32\drivers\VBox*
• VirtualBox directories: C:\Program Files\Oracle\VirtualBox Guest Additions\
• Sandbox artifacts: C:\cuckoo\, analysis.conf, analyzer.py

Registry Reconnaissance

The Windows registry contains a wealth of information about the system: hardware configuration, installed software, drivers, language settings, startup programs, and much more. For malware, this makes the registry a perfect place to look for signs of virtualization or sandboxing. A few keys are enough to expose whether the program is running inside VirtualBox, VMware, or even on a specific company’s machines.

A common target is the hardware description hive:

HKLM\HARDWARE\Description\System

When VirtualBox is installed, this hive often contains values that explicitly reference the hypervisor. One of the clearest giveaways is the string VBOX -1, which appears as part of the system description. This value is automatically written by VirtualBox to identify the virtual hardware it emulates. From the point of view of malware, finding VBOX -1 means the system is not a real physical machine, but a VM.

The code to perform this check is straightforward. Malware opens the registry key with RegOpenKeyEx, then enumerates its values with RegEnumValue, comparing each entry against a suspicious string. A simplified version looks like this:

HKEY hKey;
RegOpenKeyEx(HKEY_LOCAL_MACHINE,
             "HARDWARE\\Description\\System",
             0,
             KEY_READ,
             &hKey);

DWORD index = 0;
char valueName[MAX_PATH];
BYTE data[MAX_PATH];
DWORD valueNameSize, dataSize, type;

while (RegEnumValue(hKey,
                    index,
                    valueName,
                    &valueNameSize,
                    NULL,
                    &type,
                    data,
                    &dataSize) == ERROR_SUCCESS) {

    if (strcmp((char*)data, "VBOX -1") == 0) {
        TerminateProcess(GetCurrentProcess(), 0);
    }
    index++;
}

In practice, the sample doesn’t need to parse every value. A single hit on "VBOX -1" or "Oracle VM VirtualBox" in this hive is enough for it to assume it’s being analyzed. Similar traces exist for VMware under keys such as HKCU\SOFTWARE\VMware, Inc. or HKLM\SYSTEM\CurrentControlSet\Enum\IDE, which include references to “VMware Virtual IDE Hard Drive.”

Malware can also use registry checks for target profiling. For example, if an attacker wants the payload to run only on systems belonging to a specific company, the malware could query for a custom software key under HKCU\Software\<CompanyName>. If the key is missing, execution stops; if it exists, the malware proceeds. This ensures the malware won’t spread uncontrollably outside of its intended scope.

Legitimate software constantly queries the registry, so registry activity on its own is not suspicious. The difference lies in what is being queried. A loop over ACPI entries like HKLM\HARDWARE\ACPI\DSDT\VBOX__ or startup keys pointing to VBoxTray is unlikely to be benign. Spotting these queries during dynamic analysis is often the first sign that the malware is checking its surroundings before going any further.

Some registry keys frequently used as virtualization indicators include:

VirtualBox:

• HKLM\HARDWARE\Description\System → contains VBOX -1 and Oracle VM VirtualBox
• HM\HARDWARE\ACPI\DSDT\VBOX__
• HKLM\SOFTWARE\Oracle\VirtualBox Guest Additions
• HKLM\SYSTEM\ControlSet001\Services\VBoxService

VMware:

• HKCU\SOFTWARE\VMware, Inc.
• HKLM\SOFTWARE\Classes\Applications\VMwareHostOpen.exe
• HKLM\SYSTEM\CurrentControlSet\Enum\IDE\DiskVMware_Virtual_IDE_Hard_Drive

Services

Another place where malware often looks for virtualization clues is in the list of system services. Services are background processes that start with Windows and usually run without user interaction. Because hypervisors need extra components to integrate with the guest operating system, they install their own services during setup. For malware, those services are a reliable fingerprint of a VM.

Take VirtualBox as an example. Its Guest Additions package installs the VBoxService, which runs continuously in the background. VMware has similar components, such as VMTools or VGAuthService. On a typical user workstation you wouldn’t expect to see these, so their presence is a strong sign that the environment is virtualized.

Malware can enumerate services in several ways. Using the Windows API, the most common approach is EnumServiceStatus or its extended version EnumServiceStatusEx. These functions return a list of services along with their current state (running, stopped, disabled). Another option is OpenService, which tries to get a handle to a specific service. If the call succeeds, the service exists - and the malware has confirmation.

Here’s a simplified example using the API to check for VBoxService:

SC_HANDLE scm = OpenSCManager(NULL, NULL, SC_MANAGER_ENUMERATE_SERVICE);
ENUM_SERVICE_STATUS services[1024];
DWORD bytesNeeded, servicesReturned, resumeHandle = 0;

EnumServicesStatus(scm,
                   SERVICE_WIN32,
                   SERVICE_ACTIVE,
                   services,
                   sizeof(services),
                   &bytesNeeded,
                   &servicesReturned,
                   &resumeHandle);

for (int i = 0; i < servicesReturned; i++) {
    if (strcmp(services[i].lpServiceName, "VBoxService") == 0) {
        TerminateProcess(GetCurrentProcess(), 0);
    }
}

Of course, not all malware relies on APIs. Some simply invoke command-line tools like sc query or wmic service get name, which produce the same information. Others go directly to the registry, since service configuration is stored under HKLM\SYSTEM\CurrentControlSet\Services\.

For analysts, this means that monitoring service-related API calls or repeated sc/wmic executions can be a good detection strategy. If a sample is walking through the list of services and comparing names, it’s very likely performing an environment check before deciding whether to run its payload.

Some services that often give away virtualization are:

• VMware: VMTools, VGAuthService
• VirtualBox: VBoxService

Installed Software

Another clue malware often relies on is the list of installed programs. Every time you install software on Windows, the system leaves traces: folders in Program Files, registry entries under Uninstall, and sometimes even background services. By walking through these artifacts, malware can discover not only whether it’s in a virtual machine or sandbox, but also whether the host is a valid target.

For example, many malware analysis VMs have tools like Wireshark, Fiddler, Process Explorer, or IDA Pro installed. These tools rarely appear on a normal user’s computer, so spotting them is a strong hint that the environment is artificial. On the other side, targeted malware may look for very specific business applications to make sure it only executes inside its intended victim organization.

The Windows API offers direct access to this information. Functions like MsiEnumProducts return the list of installed MSI packages, and registry queries under

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall

reveal software names, versions, and vendors. Malware can simply loop through these entries and check for keywords of interest.

Here’s a simplified routine that walks the Uninstall key and looks for “VirtualBox”:

HKEY hKey;
RegOpenKeyEx(HKEY_LOCAL_MACHINE,
             "SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall",
             0,
             KEY_READ,
             &hKey);

DWORD index = 0;
char subkey[MAX_PATH];
DWORD subkeySize = MAX_PATH;

while (RegEnumKeyEx(hKey, index, subkey, &subkeySize,
                    NULL, NULL, NULL, NULL) == ERROR_SUCCESS) {
    if (strstr(subkey, "VirtualBox")) {
        TerminateProcess(GetCurrentProcess(), 0);
    }
    index++;
}

From the malware’s perspective, this check is very cheap: all it needs is one suspicious entry to confirm it’s in a lab. From the analyst’s perspective, it means that even installing your favorite debugging tool can become an easy giveaway. This is why many analysts prefer to use portable versions of tools, which don’t require installation and therefore don’t create registry keys or background services.

Some installed software artifacts often targeted by malware include:

• Virtualization tools: VirtualBox Guest Additions, VMware Tools
• Analysis tools: Wireshark, Process Explorer, Procmon, Fiddler
• Disassemblers/debuggers: IDA Pro, OllyDbg, x64dbg

Mutexes

Mutexes (short for mutual exclusions) are Windows objects used to manage access to resources. They’re not inherently malicious - any application can create one to prevent two processes from writing to the same file at the same time, for instance. But their presence can also become a fingerprint.

Some virtualization platforms and even certain sandboxes create mutexes with very distinctive names. For malware authors, that’s an opportunity: instead of scanning the whole filesystem or registry, a single API call can confirm whether those mutexes exist. If they do, the malware concludes it’s running in a virtual machine or under analysis.

The API of choice here is OpenMutex. Malware attempts to open a handle to a mutex with a known name. If the call succeeds, the mutex exists and the system is most likely virtualized. Here’s a simple illustration:

HANDLE hMutex = OpenMutex(MUTEX_ALL_ACCESS, FALSE, "VMwareGuestCopyPasteMutex");
if (hMutex != NULL) {
    TerminateProcess(GetCurrentProcess(), 0);
}

This technique has the advantage of being lightweight and stealthy. Checking a single object doesn’t raise the same level of noise as looping through processes or registry hives. For analysts, that makes it trickier: mutex checks often appear as a single API call buried in the code, easy to miss unless you know what to look for.

The key takeaway is that not every mutex you see is suspicious. But if the malware tries to open one with a name tied to a hypervisor or a sandbox, that’s a strong signal that it’s performing environment checks.

Some mutex names frequently used as virtualization indicators include:

• VMware: VMwareGuestCopyPasteMutex, VMwareGuestDnDDataMutex, VMToolsHookQueueLock
• VirtualBox: VBoxService, VBoxTray

Named pipes

Pipes are another Windows object that malware can exploit to figure out its environment. A pipe is simply a communication channel between processes, but hypervisors often create their own named pipes to handle guest–host interaction. These pipe names can be very distinctive, and their presence is an easy way for malware to confirm it’s running inside a VM.

Detecting them doesn’t require anything exotic. Malware can call functions like CreateFile or CallNamedPipe with the name of a target pipe. If the call succeeds, the pipe exists and the system is almost certainly virtualized. The check is both quick and reliable.

Here’s a simplified example:

HANDLE hPipe = CreateFile("\\\\.\\pipe\\vgauth-service",
                          GENERIC_READ | GENERIC_WRITE,
                          0,
                          NULL,
                          OPEN_EXISTING,
                          0,
                          NULL);

if (hPipe != INVALID_HANDLE_VALUE) {
    TerminateProcess(GetCurrentProcess(), 0);
}

From the analyst’s perspective, this is similar to the mutex trick: one API call can be enough for malware to decide whether to continue. And because pipes are a normal part of Windows interprocess communication, the activity doesn’t stand out unless you know the names to watch for.

A good way to see these objects in your own lab is to use Pipelist, a Sysinternals tool that enumerates all named pipes on a system. Run it inside a virtual machine and you’ll usually spot entries that give away the hypervisor.

Some common pipes that betray virtualization are:

• VMware: vgauth-service, Vmmemctl
• VirtualBox: VBoxTray, VBoxTrayIPC, VBoxGuest, VBoxVideo, VBoxMouse, VBoxMiniRdr

Devices & Drivers

Every Windows system maintains a set of devices and drivers that represent its hardware and allow the OS to communicate with it. A device might stand for something physical, like a USB controller or disk, while a driver is the software component that manages that hardware. Hypervisors need their own drivers and devices to handle things like shared folders, mouse integration, and virtual graphics. As a result, both VMware and VirtualBox leave behind very specific driver and device objects that malware can spot.

These objects are managed by the Windows Object Manager. To enumerate them, malware often uses the native APIs NtOpenDirectoryObject and NtQueryDirectoryObject. The first call opens a handle to a directory (for example, \\Driver), and the second walks through its contents, returning the list of driver objects available on the system. By scanning this list, the malware can look for telltale names that indicate virtualization.

A simplified routine might look like this:

HANDLE hDir;
UNICODE_STRING dirName = L"\\Driver";
OBJECT_ATTRIBUTES attrs;

InitializeObjectAttributes(&attrs, &dirName, OBJ_CASE_INSENSITIVE, NULL, NULL);

// Open the \Driver directory
NtOpenDirectoryObject(&hDir, DIRECTORY_QUERY, &attrs);

BYTE buffer[1024];
ULONG context = 0, returnLength;

while (NtQueryDirectoryObject(hDir,
                              buffer,
                              sizeof(buffer),
                              TRUE,      // single entry per call
                              FALSE,     // continue enumeration
                              &context,
                              &returnLength) == STATUS_SUCCESS) {

    if (strstr((char*)buffer, "VBox") || strstr((char*)buffer, "vmhgfs")) {
        TerminateProcess(GetCurrentProcess(), 0);
    }
}

This check is low-level but very effective. A driver named VBoxGuest or vmhgfs will almost never exist on a physical machine, so its presence is a strong indicator that the system is virtualized. Malware can perform the same operation on the \\Device directory to enumerate device objects, which can be equally revealing.

For analysts, this technique is important because it often looks like “normal” OS interaction. Unless you know the specific driver names that hypervisors install, it’s easy to miss why the malware is poking around \\Driver or \\Device. Seeing calls to NtOpenDirectoryObject and NtQueryDirectoryObject in a sample should immediately raise the possibility that it’s checking for virtualization artifacts.

Some common drivers and devices include:

VMware drivers: vmhgfs, vmci, vmmouse, VMMemCtl
VirtualBox drivers: VBoxGuest, VBoxMouse, VBoxSF, VBoxVideo, VBoxWddm

VMware devices: VMCIGuestDev, VMCIHostDev, vmmemctl
VirtualBox devices: VBoxGuest, VBoxMiniRdr

Usernames & Hostnames

Another simple but effective check malware often performs is looking at the system’s usernames and hostnames. Automated sandboxes usually rely on default configurations: accounts called Administrator or User, machines named Test, Lab, or Cuckoo. These values are quick to set up and easy to randomize, but in practice many environments leave them unchanged. For malware, that’s a cheap and reliable fingerprint.

Enumerating this information doesn’t require anything exotic. Windows provides APIs and environment variables that reveal the current user and the computer name. A sample can compare these strings against a small list of suspicious values and decide whether to continue. For example, spotting a hostname like TequilaBoomBoom - once used by VirusTotal’s sandbox - is enough for many malware families to shut down immediately.

Here’s a simplified check:

char username[256];
char hostname[256];
DWORD size = 256;

GetUserName(username, &size);
GetComputerName(hostname, &size);

if (strcmp(username, "Administrator") == 0 ||
    strcmp(hostname, "Cuckoo") == 0) {
    TerminateProcess(GetCurrentProcess(), 0);
}

This kind of profiling doesn’t just apply to sandboxes. In targeted attacks, malware may also look for specific patterns used inside an organization - for example, checking if the hostname follows a corporate convention like CORP-WS-123. If the pattern doesn’t match, the malware may refuse to run, ensuring it doesn’t spread outside its intended scope.

For analysts, this means that generic usernames and machine names make a VM stick out. Configuring them with realistic values, or at least non-obvious ones, helps reduce the risk of immediate detection during detonation.

Some common values malware checks for include:

• Usernames: Administrator, User, John
• Hostnames: Test, Lab, Desktop, Workstation, Cuckoo, TequilaBoomBoom

Locale & language

Malware doesn’t only care about whether it’s in a VM or sandbox - sometimes it wants to know where the victim is located. Language and locale settings give away both geography and user preferences, and attackers can use this information to decide whether to deploy their payload.

There are several reasons for this. Some malware families avoid infecting systems in certain regions (for example, Russia or other CIS countries) either for geopolitical reasons or because the operators want to avoid drawing local law-enforcement attention. Others may be designed to target users in a specific language group, so checking the system’s keyboard or UI language is an easy filter.

Windows exposes these settings through multiple APIs. GetKeyboardLayout returns the current keyboard language, while GetKeyboardLayoutList enumerates all installed layouts. For UI languages, functions like GetUserDefaultUILanguage and GetSystemDefaultUILanguage reveal the display language at both user and system levels. Finally, locale information - things like number formats, currency, and date styles - can be obtained with GetSystemDefaultLCID or GetUserDefaultLCID.

A simplified check might look like this:

HKL layout = GetKeyboardLayout(0);
if (layout == 0x419) { // Russian
    TerminateProcess(GetCurrentProcess(), 0);
}

Here the malware calls GetKeyboardLayout and compares the result against the identifier 0x419, which corresponds to Russian. If that’s the active layout, the program terminates immediately instead of running its payload.

This technique can be extended further: some malware inspects locale-related registry entries, or even looks at how the system formats dates and currency. All of these are indirect signals that reveal the user’s location.

For analysts, this means that a sample which seems to do nothing may not be broken - it may simply have detected that the system isn’t in its intended target region. Running the sample with different language packs or locale settings often makes the behavior visible.

Some common Windows language identifiers include:

• 0x402 → Bulgarian (Bulgaria)
• 0x409 → English (United States)
• 0x407 → German (Germany)
• 0x419 → Russian (Russia)

OS version checks

Sometimes, before a malware sample executes its payload, it needs to know exactly which version of Windows it has landed on. This check is not always about evasion - often it’s about compatibility. An exploit written for Windows XP may crash a Windows 7 machine, and a payload that relies on Windows 7 internals may fail on Windows 10. To avoid instability that could reveal its presence, malware checks the OS version first.

A famous example is Stuxnet. Its operators knew their target environment ran Windows XP, so the worm only deployed its exploit chain on that version. On newer systems, it would simply stop, making it appear inert to analysts who tested it outside the intended OS.

Windows provides several APIs for version checks. The most common are GetVersion and GetVersionEx, which return the major and minor version numbers of the running OS. With those values, the malware can decide whether to proceed.

Here’s a simplified snippet:

OSVERSIONINFO info;
info.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);

if (GetVersionEx(&info)) {
    if (info.dwMajorVersion == 5 && info.dwMinorVersion == 1) {
        // Windows XP
        RunPayload();
    } else {
        TerminateProcess(GetCurrentProcess(), 0);
    }
}

In this example, the malware only runs its payload on Windows XP (5.1). If the version doesn’t match, it quietly terminates. To an analyst running the sample on a modern Windows 10 VM, the program would appear to “do nothing,” when in reality it was performing a strict compatibility check.

This is why OS version enumeration is relevant for analysis: a sample that seems inactive may not be evasive in the usual sense, but simply unwilling to risk crashing the wrong platform. Recognizing these checks helps distinguish between malware that is hiding and malware that is designed for a very specific environment.

Some common OS version values returned by GetVersionEx are:

• Windows 10 / Server 2016: 10.0
• Windows 8.1: 6.3
• Windows 7: 6.1
• Windows XP: 5.1