Original translation tohttps://www.xenserver.com/story, The birth of Xen: A journey from XenoServers to Cloud Virtualization.Editor’s Note:This article narrates the entire process of the development of Xen virtualization technology from an academic research project at the University of Cambridge to a core component of global cloud computing infrastructure, showcasing a paradigm of technological innovation, open-source collaboration, and business strategy integration. Xen originated from the XenoServers project in 1999, addressing the performance bottlenecks of the x86 architecture through paravirtualization technology, and was first open-sourced in 2003, attracting participation from organizations such as Microsoft and Intel. With the adoption by Amazon and Google, Xen became a key technology for building cloud services. In 2005, XenSource was established to promote commercialization; in 2006, a partnership was formed with Microsoft, gaining market recognition; and in 2007, it was acquired by Citrix, further expanding its influence.This is a true story of the integration of technological ideals, open-source spirit, and business wisdom, inspiring technologists to not only innovate but also to have the capability to drive implementation.
1999
Ian Pratt Introduces the Birth of XenThe history of Xen began in 1999 with the launch of the XenoServers project at the Computer Laboratory of the University of Cambridge. Dr. Ian Pratt and several graduate students proposed the creation of a server infrastructure spread across the internet, where any properly authenticated individual or organization could purchase resources on these machines for computation or service execution, dynamically adjusting resource usage as needed. The project was named “XenoServers,” derived from the ancient Greek word “xenos”—a stranger, whom you might provide food and shelter but not necessarily trust. The first paper on the project was presented at the ACM HotOS conference in 1999, titled “Xenoservers: Accountable Execution of Untrusted Programs.”
A More Powerful Platform
Initially, the project considered using a Java virtual machine to run customer-provided code, but soon realized that if it could run any existing application without forcing everything to be rewritten in Java, the platform would be much more powerful. Using a hypervisor and encapsulating applications and their chosen operating systems in virtual machines would be the ideal way to achieve this goal, but at that time, virtualization on the x86 architecture relied on instruction emulation and binary rewriting, which posed performance challenges.
Thus, a method was developed to allow operating systems to be ported (“paravirtualization”) to run on the hypervisor, enabling applications to run without modification. Keir Fraser was the graduate student who seized this idea and turned it into reality, building the core hypervisor and first porting Linux 2.2. This hypervisor was named “Xen,” an abbreviation of its parent project name.
A key goal of Xen was to allow virtual machines from different customers to run concurrently on the same hardware, achieving strong isolation to ensure that one customer could not steal or corrupt another customer’s information, and that each customer received all the resources they paid for. We had a vivid design requirement: to be able to sell services simultaneously to Coca-Cola and Pepsi, and to ensure they felt secure about this arrangement. This is at the core of Xen’s design.
2003
First Complete Open Source Release
In the following years, as more people recognized the significant potential of the project in both academia and the real world, the number of students and research assistants involved in the Xen and XenoServers projects continued to grow. Commercial research labs such as Intel Research and HP Labs also began to fund and participate in the project’s development.
The first complete open-source release of Xen was completed in 2003, with the hypervisor using the GPLv2 license, while the internal components of the operating system were licensed under BSD to facilitate the porting of proprietary operating systems. Microsoft Research also contributed to the project, with Paul Barham leading the prototype porting work for Windows XP.
Later in 2003, a key research paper titled “The Art of Virtualization with Xen” was published at the top ACM conference SOSP. The momentum around the development of Xen continued to grow, with several internet service providers beginning to adopt Xen/XenoServers to offer commercial hosting services to customers, fully leveraging the project’s capabilities in computation, networking, and storage virtualization.
2004
An Excellent Academic Research Platform
In early 2004, Christian Limpach ported the NetBSD operating system to Xen and subsequently joined the Cambridge team, becoming a key contributor supporting Keir. Under Pratt’s leadership, the open-source community around Xen continued to grow, with companies like RedHat, SuSE, Sun, IBM, and AMD joining the community and contributing, either by porting their operating systems to Xen or by porting Xen to their chips and hardware.
The project recognized early on that extending the x86 architecture could simplify virtualization tasks, thereby improving performance, reducing the amount of privileged code required, and lowering the attack surface. Close collaboration with Intel led to the development of the Xen HVM mode, which utilized Intel’s VTx extensions. The HVM mode avoided the need for partial paravirtualization modifications to the operating system, making it easier to run unmodified operating systems, while still using paravirtualization techniques in scenarios where performance could be improved and the attack surface reduced. HVM mode later also added support for AMD-V.
In addition to its increasing role in commercial products and services, Xen also became an excellent academic research platform, not only at the University of Cambridge but also at other universities and research labs around the world, with many research outcomes later applied to actual products. For example, the Xen architecture was extended to a concept called driver domains, which removed the privileged code used to control IO devices and isolated it in virtual machines, enabling seamless recovery after driver crashes or real-time software updates (OASIS 2003, “Secure Hardware Access Using Xen Hypervisor”). Steve Hand led further separation of the Xen control plane to enhance security. Xen was also used to research real-time migration of virtual machines between different machines for load balancing and hardware maintenance (Usenix NSDI 2005, “Real-Time Migration of Virtual Machines”). It was also used to implement techniques such as deterministic replay and checkpointing to support high-performance hardware fault-tolerant computing, as well as advanced “reverse debugging” techniques for distributed systems.
Becoming an Important Infrastructure Component
By mid-2004, Xen was gradually becoming an important infrastructure component. Both Amazon and Google were using Xen internally and preparing to launch what we now call “Infrastructure as a Service” cloud platforms, maximizing the potential of the XenoServers model. In addition to companies building products and services based on Xen, an increasing number of forward-looking enterprises also saw the potential of Xen to transform data centers, enabling them to achieve infrastructure virtualization and even outsource infrastructure to other data centers.
Members of the Xen team were increasingly invited to provide consulting services to support these plans, and upon discovering opportunities to create enterprise-level packaged products, the main contributors to Xen founded a company called XenSource. It was not easy to secure funding for the company in the UK and Europe, as the business model around open-source software had not yet been fully established. This prompted Pratt to reconnect with an old university friend, Simon Crosby, and head to Silicon Valley, where they encountered strong interest, with XenSource’s Series A funding provided by Kevin Compton (KPCB) and Nick Sturiale (Sevin Rosen).
2005
Simon Crosby Continues the Story of 2004
In the fall of 2004, I was at Intel trying to secure research funding for Ian’s team but faced significant resistance. It turned out that Intel was considering investing in VMware… I met with Ian and other members of his team (a total of 7 co-founders) at a meeting in San Francisco, and we decided to start a company. Moshe Bar also joined us, although I didn’t know him at the time, but he had also been in communication with Ian. Our founding CEO Nick Gault was also present. Morgan Stanley showed genuine interest in us as a potential client, while our open-source codebase gained significant attention.
Between Christmas and New Year in 2004, we visited some investors on Sand Hill Road, and the company was eventually founded in early 2005, securing $8 million in funding. At that time, all our energy was focused on building the open-source codebase and community. Xen 2 had been released, and the community was working hard on developing Xen 3. To be honest, we didn’t think much about how XenSource would make money initially; the original idea was to provide management tools and let distributions like Red Hat and Suse integrate the hypervisor into their Linux products. We were not able to virtualize Windows at that time because Xen was a paravirtualizing hypervisor that required modifications to the guest operating system, and Microsoft was not willing to do that!
The first year of the company’s existence was entirely focused on open-source development, but we soon realized that large distributions also wanted a piece of the pie: they announced plans to provide management tools, so we faced competition even in the open-source space. By the end of the first year, due to our failure to clearly articulate a business plan, we lost Moshe and our founding CEO. Fortunately, we met Peter Levine and Frank Artale through Accel Ventures, who joined us and brought the necessary business thinking: at this point, we decided to compete directly with VMware. We also completed a Series B funding round, which I believe was $25 million. Our product was a virtualization platform similar to ESX, initially capable of virtualizing only Linux, but when Intel introduced CPU virtualization extensions, we were also able to virtualize Windows and added that option. Subsequently, we developed a clustering product, which I believe is superior to VMware’s virtual infrastructure.
2006
Microsoft Establishes Partnership with Xen
Throughout this process, we were always pondering one question: how can an open-source company survive when all the code is open? We had two real advantages: first, the complex management software for the hypervisor was written in OCaml—a relatively obscure but type-safe language that was well-suited for this task. Second, Frank Artale had strong connections at Microsoft, which was building a hypervisor for Windows but needed a third party to assist with the open-source and Linux parts to avoid contaminating their developers with open-source.
We agreed to do this work for them, establishing an excellent partnership, with Microsoft recommending our products whenever possible instead of VMware. As a competitive hypervisor recommended by Microsoft, we gained the credibility needed for startup revenue. Our collaboration with Microsoft began in the fall of 2006.
2007
Citrix Acquires XenSource Inc
It seemed that Microsoft was not the only player dissatisfied with VMware. Citrix initially partnered with VMware, but when VMware entered the desktop virtualization market, competing with their application virtualization business, Citrix became disillusioned. By mid-2007, given our partnership with Microsoft in virtualization, Citrix viewed us as a potential acquisition target. We would become a platform friendly to Microsoft, aligning with their collaboration in application virtualization. Citrix acquired us in the fall of that year.
Meanwhile, Amazon began using Xen to build its cloud services starting in late 2005. Ultimately, with the deployment of millions of virtual machines, they became the largest user and arguably the biggest beneficiary of the technology. It is no exaggeration to say that Xen built cloud computing. Our collaboration with Amazon was highly productive, although it was entirely based on open-source usage rather than commercial collaboration. They were able to virtualize Windows because they eventually adopted a new generation of Intel platforms with virtualization extensions.
…..To be continued….
Postscript: Citrix Systems was jointly acquired by private equity firms Vista Equity Partners and Evergreen Coast Capital in September 2022, with a total transaction value of approximately $16.5 billion. This acquisition transformed Citrix from a public company into a private entity and merged it with TIBCO Software, a subsidiary of Vista, to form a new Cloud Software Group (CSG) enterprise software group. This group serves over 400,000 customers globally across more than 100 countries, including 98% of the Fortune 500 companies. This privatization not only enhanced Citrix’s strategic flexibility but also accelerated its transition to SaaS and cloud services.
The Cloud Software Group has one of its five global R&D centers located in Nanjing, China.