While I am away on vacation, I have invited Joel Wampler to be a guest author on the ElasticVapor Blog, hopefully something I will be doing more frequently in the future. Joel is a Systems Architect at Standing Cloud, Inc., a startup that will be soon launching software and services that facilitate application deployment and management for on-demand cloud services. In this post, he points out that last week's Linux kernel vulnerability has some extra implications for IaaS and infrastructure focused cloud computing as well as some simple steps to fix these vulnerabilities.
Issues like this are clear examples of something we at Enomaly have been saying for some time. Like it or not cloud computing brings unique security risks with it, which need to be specifically managed and mitigated. This latest Linux Kernel vulnerability is another prime example of the challenges facing cloud service providers and their customers. To help reduce this risk later this year Enomaly will deliver the first generation of our security technology, which will help customers to achieve a level of security in the cloud equivalent to what they have inside their data centers today.
Thank you Joel for taking the time to bring awareness to the new risks facing cloud users.
---
Last week, a Linux kernel vulnerability that allows for local privilege escalation through a NULL pointer dereference was announced. Many of the major Linux distributions are still working to provide updated kernels, and a few already have. Once updated kernels are released, applying the patches should be straightforward. But for systems running in the cloud, additional complexities and delays may arise.
Most providers of on-demand cloud servers require the use of custom kernels, which are tuned for the provider's specific virtualization implementation. These custom kernels significantly change the upgrade path, and may even affect the short-term workarounds provided by the upstream distribution.
For instance, the Ubuntu bug report for this issue states the following:
Ubuntu 8.04 and later have a default setting of 65536 in /proc/sys/vm/mmap_min_addr. When set, this issue is blocked.
However, if a system is running Ubuntu 8.04 on Amazon EC2, the underlying kernel is likely based on a Fedora Core 8 Xen kernel. This is one of the kernels Amazon provides to those who create boot images for their service, and most such images use this kernel regardless of the distribution running on top of it. Thus the default setting of 65536 cannot be relied upon; and worse, this proc setting does not even exist in the Fedora kernel, so there is no way to repair the image to match this workaround.
When a cloud provider does eventually provide updated kernels, applying them to running cloud servers will also present challenges that don't exist with bare iron. The upgrade method will vary greatly between providers. It could be as simple as contacting support and asking them to make the switch, or it could be a tedious process involving re-bundling and re-registering images. Whatever it may involve, it is certainly not as easy as just applying the latest kernel packages and rebooting.
Kernel bugs of this severity don't arise too often. But a cloud provider's custom kernel is the one piece of software that users of on-demand cloud servers cannot simply replace. This vulnerability sheds light on an area that hasn't been completely hashed out yet by on-demand providers; they need to be more proactive in both distributing information about kernel security issues, as well as documenting image and instance upgrade procedures once a fix is available.
As to this particular vulnerability, the workaround originally provided by RedHat and also by Ubuntu in their bug report, seems to be the most universal means of mitigation until new kernels are released (whether or not the server is in the cloud).
Simply enter the following at the command line as root:
cat > /etc/modprobe.d/mitigate-2692.
conf << EOM
install pppoe /bin/true
install pppox /bin/true
install sctp /bin/true
install bluetooth /bin/true
install irda /bin/true
install ax25 /bin/true
install x25 /bin/true
install ipx /bin/true
install appletalk /bin/true
EOM
This will prevent any modules known to be affected by the vulnerability from being loaded, which should prevent the exploit from being successful. Fortunately, most of these modules are not widely used on cloud servers. Of course, if your application requires one of these modules, you may need to find another mitigation or to run insecurely until other fixes are available.
