There has been recent chatter in comp.os.vms around malware targeting OpenVMS. There has been some over the years. But much like malware targeting Mac OS X and Mac OS X Server, the malware has been rare, and rarely seen in the wild.
Andy Goldstein of OpenVMS Engineering wrote a proof-of-concept virus some years ago. It was never released into the wild. There have been various other similar proof-of-concept virii implemented over the years, though those too have not seen release.
Steve Reece reports having encountered a DCL-based virus in the wild, circa 1994.
An ill-fated mail distribution was attempted at U Mass Amherst, reportedly by someone claiming to be T.J. Bryce. These messages were distributed at Amherst circa 7-Oct-1990 and 8-Oct-1990. The messages contained an executable, either claiming to be a game, or requesting help with the image.
And there was a self-replicating DCL-based virus released a decade or so back.
Here is an example self-replicating DCL virus, as posted to comp.os.vms back in 1994 — with one tweak made to effectively completely disable it:$!XYZZY
$ OPEN/READ FILE 'F$ENV("PROCEDURE")'
$ OPEN/APPEND ELIF 'F$SEARCH("*.COMMAND")'
$ LBL0: READ/END=DONE FILE REC
$ IF REC .NES. "$!XYZZY" THEN GOTO LBL0
$ LBL1: WRITE ELIF REC
$ READ/END=DONE FILE REC
$ IF REC .NES. "$!YZZYZ" THEN GOTO LBL1
$ WRITE ELIF REC
$ DONE: CLOSE FILE
$ CLOSE ELIF
$!YZZYX
Several DECnet worms have been active many years ago, including the WANK (Worms Against Nuclear Killers) worm. There were also several worms — or worm-like mechanisms — propagated within DIGITAL and its large DECnet network.
Password grabbers were seen around, though the optional secure server mechanism (eg: SET TERMINAL /SECURE_SERVER /[NO]AUTOBAUD) and changes to terminal device protections effectively blocked these. I've seen a few trojan horses over the years, too.
It's quite certainly feasible to create a self-replicating infection on OpenVMS, and it's quite easy for these to propagate within a UIC group, or within shared resources. If there are mixtures of privileged and unprivileged usernames within the same UIC group, one of these can become fully privileged — most look to infest LOGIN.COM or SYLOGIN.COM. For DCL, less than twenty lines of code are minimally required. Image-level propagation isn't that much more difficult than a DCL-based propagation; you could conceivably target arbitrary images for nefarious extensions. And once a privileged username has been infested, the operating system itself can be targeted.
Classic Windows malware tends to replicate through one of several means including spoofing the file type, and tends to propagate through mail or through port-level attacks. And you can see infections spreading across files within a system, similar to the DCL-based or image-based attacks feasible on OpenVMS.
Why is it tough to target OpenVMS?
So why isn't a DCL virus everywhere?
OpenVMS is primitive, bluntly. It's more difficult to mount a successful attack against a comparatively constrained surface area.
OpenVMS does not include a “magic” mechanism and — outside of DECwindows — does not have a click-to-act mechanism. “Magic” sniffs the file contents, seeking to determine its internal organization and structure. This mechanism has been successfully targeted on the Windows platform on several occasions.
OpenVMS does not have a Microsoft .NET (OLE-style) interconnection mechanism, allowing application chaining and trivial application invocations within applications. In particular, the OpenVMS MAIL application is primitive, and it is this primitiveness and this lack of chaining that prevents widespread infections — you can't invoke an executable or a DCL procedure from within MAIL, you have to extract it (/NOHEADER) and then run it. And the loose integration with MIME means you either need a DECnet link (SEND/FOREIGN) or you need to extract it, MIME it, and then invoke it. There are few packages that include macro-level or embedded languages, such that opening a document triggers executable code. Of course, spoofing the source address of a DECnet or SMTP mail message — to appear to be a trusted source — is fairly easy.
OpenVMS does not have a widespread download mechanism, nor icons or gadgets or other decorations. There is insufficient community infrastructure around file exchange to support user-downloaded and user-installed malware — sure, download sites exist, but arbitrary freeware packages are just not high-bandwidth downloads, and such packages are typically not installed on production servers. The kids in the household just aren't installing “free smilies” or free music download software or other such onto the local OpenVMS box. This reduces exposures to buggy downloads, and to downloads containing malware.
There have been proof-of-concept Javascript infections (virii and worms) that could likely infest OpenVMS; these use the Javascript provided within a web browser to perform any of various activities either locally, and these can be fully capable of opening remote connections and attempting propagation. Several Javascript virii and worms active in the wild target platform-specific security weaknesses; vulnerabilities not present within OpenVMS. See Nimda, et al. But by default, OpenVMS does not include a web browser. (Noticing a pattern?)
As for more generalized network security and defenses against remote attacks by malware, there have been various port-level exposures over the years, with the IP Ping Of Death denial-of-service attack being one of the more notable. There have been others. One of the more prevalent approaches of late involves little more than sniffing the network — telnet, LAT and DECnet all transmit information in the clear, and this can include username and password pairs. These have not been targeted in any widespread fashion, though it would not surprise me that there have been attacks. OpenVMS servers also tend to have comparatively few services and accordingly few ports open, as compared to other operating system platforms. This means there are fewer potential ways into the box.
Direct attacks are occasionally feasible. The direct approach of increasing privileges through a security vulnerability can arise and there have been paths toward higher privileges over the years, with the GRPNAM attack being one of the more notable. There was a coding error in a VAX/VMS V4 version of a system service that allowed unfettered write access into SYSUAF. These cases have been patched. There have been others.
Can OpenVMS system security be breached? Historical evidence indicates there have been vulnerabilities.
Is breaching OpenVMS security easy? In general, no.
So why is OpenVMS secure? There are few OpenVMS boxes around and they tend to be isolated clumps, and common vectors such as the integrated mail package are primitive transmission vectors at best, and the numbers of network-level services active on a typical OpenVMS server tends to be low. OpenVMS itself seeks to keep security within the operating system and the kernel, and classic OpenVMS applications tend not to delegate security.
Probably the biggest potential target for vulnerabilities and for malware propagation toward or into OpenVMS that presently exist involve browsing directly from OpenVMS systems. Or ill-written privileged application software.
If you can locate copies, look for EDP Auditor Journal, Volume 1, 1993. This issue reportedly contains the following articles:
- “Open VMS VAX Security Architecture” by Donald Holden
- “Technical Implementation of VAX/VMS Security” by Ray Kaplan and Joe Kovara
- "Good Security Practices in a VAX/VMS Environment” by Todd J. Whiting and Robert A. Clyde
- “Auditing VAX/VMS Systems” by Joseph L. Oringel
- “Security Considerations for Interconnected VAX/VMS Systems” by Ray Kaplan and Joe Kovara
- “New Security Features in VAX/VMS V6.0” by Lawrence J. Kilgallen
Should you find this issue, do let HoffmanLabs know.
What to Do?
How do you secure an OpenVMS box? Well, what do you have running on it, and how much is the information or run-time access to the server itself worth to somebody? (To an attacker, your information may be worth far less than simply having unfettered access to a high-bandwidth network server, or network-accessible storage.) The determination of value tells you how much to spend on security.
As for the typical suggestions, shut down all unnecessary network services.
Keep current on your OpenVMS ECOs — subscribe to the OpenVMS.org patch notification list, or the HP equivalent.
Don't install software you don't need.
Use only encrypted network protocols.
Monitor access failures and attack patterns, and watch for security probes.
Ensure all network-facing applications — clients and servers — are maintained to current. Mozilla, ftp, php, Apache and other such network-facing applications can have vulnerabilities and can be targeted, irrespective of the software platform.
Consider maintaining an MD5 database of all system files, preferably itself either encrypted or otherwise protected.
Monitor outbound network traffic, including IRC traffic and SMTP mail.
Limit general access to your OpenVMS server to those users minimally required for correct operation, and consider performing browsing, email and other operations tasks from systems other than production servers.
Run John The Ripper or another password-cracking tool against your password database. Consider smartcards or token-based authentication, and consider a site-specific password filter that increases the required complexity of user-specified passwords. Look for embedded passwords in your system and your software, and migrate to Kerberized security or toward proxies or — since embedded passwords are basically worthless — get rid of the password entirely.
Provide information on defending your organization against security attacks made through social engineering; in most cases, this is how you will be attacked. Not through technology, but through telephone or email contact.
Consider a HoffmanLabs security review.