VRTK - From then to now to the future

Virtual Reality Toolkit

VRTK is a productive toolkit for rapidly building virtual reality experiences by providing easy to use solutions to common problems. VRTK focuses on two main areas of help for developers, interactions and locomotion techniques offering a multitude of ways of solving these common problems.

The goal behind VRTK is to bring as many creative people from as diverse backgrounds as possible to try and solve the common problems that the new medium of virtual reality brings. The faster solutions to these problems can be built, tried and tested will help accelerate the evolutionary process to find out what works and what doesn't work, because evolution is just a million mistakes until a bit of one thing works.

The way in which VRTK empowers such a large participation is to be totally free and open source (under the MIT license) to anyone who wants to use it for any reason they want to use it for, whether it's for learning the ropes of VR development, if they're using it to make the next latest VR game or even if they're a commercial entity making simulation solutions.

By making it completely free for anyone to use or to build for and with means the barrier for entry into VR development has been lowered massively so those who want to turn their creative and wild ideas into a reality can have that opportunity with VRTK.

From humble beginnings

VRTK isn't built by one of the leading VR corporations nor does it come out of some hot start up from silicon valley, in fact it was started by one person, in their bedroom in the United Kingdom because there was pretty much nothing out there that made building for virtual reality easy.

Harvey Ball, better known to the VRTK community under the name of Theston E. Fox wanted to try building something in Unity3d for the HTC Vive back in April 2016, with only a couple of months of Unity development experience under his belt and a brand new HTC Vive, he set out to build something of interest and fun in VR.

Having a career in web development set Harvey off to search on the web for useful resources, tutorials and videos of how to build stuff in Unity3d for virtual reality. It was a massive surprise to find very little out there in the way of help and instruction for any budding developer wanting to learn how to build something. The next stop was Valve's SteamVR plugin for Unity3d, which made it easier to get a virtual reality camera and controllers into a scene but how could stuff be touched, picked up or used? How could you get around a virtual scene with ease? A few games had already showcased fun and innovative ways like Valve's The Lab and Owlchemy Labs Job Simulator but there was seemingly nothing out there that helped beginners learn how to do this stuff.

There were a few example scripts that came with the SteamVR plugin that kind of showed off how to grab things and how to teleport around with a laser pointer attached to a controller but they were in no way something you could simply just drag and drop. If this was all beginners had to work with then developing for virtual reality was to become a very quiet place where only a select few would be able to turn their creative ideas into something others could play with.

This bothered Harvey as he not only came from a world of web development where free and open source resources are plentiful making it much easier for people to get started, but he also saw every VR headset owner yearning for some “killer” app and could see nothing more fun than building your own VR experiences. A throwback to the early days of home computing in the 80s where the emergence of video games hadn't risen from the “computer programmers” of the day, those folks in white lab coats feeding punch tape into computers, instead it had come from people at home, with a passion and a personal computer in their bedroom tapping out lines of BASIC to create some of the weirdest experiences of the time that would eventually become legendary in computing history.

Virtual reality was now at a point where if it was accessible to enough people then it could be that whole revolution again, where people just needed an idea and a way of expressing themselves in this new virtual medium that they could turn their wild and wonderful ideas into something that other people could enjoy. The origins of VRTK were born and the most basic of scripts arose.

One script to rule them all

Over a weekend in April 2016, Harvey set out to take all of the knowledge gleaned from the SteamVR plugin scripts and try and turn it into something that could make it easy for others to build something in VR. The script, which only took about 2 to 3 hours to write was one script for Unity3d that was dragged and dropped into a scene with the SteamVR camera rig and immediately gave the ability to shine a laser pointer within the scene and teleport to where ever the tip was pointing and also gave the ability to pick stuff up using a rudimentary fixed joint based grabbing system. This was great, someone could build a scene and with one quick drag and drop of a free script could move around their scene and pick stuff up and throw it around. The next step was to share it with the world, so on an unknown Youtube channel with no more than 100 subscribers and hardly any regular views appeared a video of how to use this VR script.
https://www.youtube.com/watch?v=6uYaK_T46z0

After a couple of days the video had hit thousands of views with obviously a lot of people in a similar situation eager to find tutorial content of how to build stuff for VR. It was quickly realised that if there was such a need and desire for this sort of content, then this basic and flimsy script was not the best way for people to move forward. It was far too limiting, it was way too coupled (meaning one script did everything so customising it would be a total pain) and it wasn't really something people could work on together in a community.

Welcome to the SteamVR Unity Toolkit

After the success of this original single script, the SteamVR Unity Toolkit was born, which was a more concerted effort to try and build a reusable and extensible collection of scripts that made building for the HTC Vive easier and quicker for not only seasoned developers but also for complete beginners who wanted to try but didn't know if they could do it. The SteamVR Unity Toolkit was a very appropriate name as it was basically a toolkit of scripts built in Unity3d that helped out when using the SteamVR plugin offering a collection of solutions such as teleporting, pointers, grabbing, touchpad locomotion. As it was all completely free and open source, it started gaining traction with people building stuff that was of interest to them in VR, some of these projects would become some of the most well known games in VR.

It was an age of the wild west of virtual reality development, no one really knew what problems would be thrown up by the medium and no one really had the answers for the solution. The SteamVR Unity Toolkit became a GitHub repository that people could share their ideas for solutions by contributing to the codebase and getting their ideas out to other people to try and make it easier for other people to build their own VR experiences and games. In fact, the developer behind QuiVR contributed a number of cool features to VRTK especially the bow and arrow example scene which inspired a number of fun bow and arrow games to be built by budding developers.

As the number of developers using the SteamVR Unity Toolkit grew, it became harder and harder to help out people with their individual problems. When there was only a handful of people using it, it was easy enough to jump on a Skype call with someone and sort out their issue, but when there's around 1,000 people using something, this is never going to end well.

No more Slacking around

The community behind the toolkit had grown rapidly in such a short amount of time and they needed somewhere that they could freely communicate and incubate ideas together. The solution was a Slack channel that anyone could join and contribute to, get help from and chat about ideas that could eventually become features of the toolkit for other people to use.

The Slack channel is still the heart of the community today with over 4,500 people worldwide working through problems and sharing ideas of how to make virtual reality experiences more interesting for the audiences out there. This became a place where people were forming real community bonds, online friendships or partnerships into new ventures of building some really cool VR games.

People in the community felt so passionate about sharing their ideas, it cemented the SteamVR Unity Toolkit as being the tool of a community effort rather than the work of one person, which just helped it grow at a faster and faster rate with more and more experiences being created.

It had grown to such a level that it had been noticed by some of the seasoned VR companies with Oculus reps asking what they could do to get it to work with their headset. Oculus were kind enough to provide a free Oculus Rift and Touch controller pack to allow Harvey to get the toolkit working on a headset other than the Vive. Within a few days it was now a multi headset toolkit with the added benefit that if something was built to work on the Vive it would now also work seamlessly enough with the Oculus Rift too. The toolkit had now also become an SDK abstraction layer that was sorely missing from the Unity3d product.

There was a small problem now with the toolkit working on the Oculus Rift, the name. SteamVR Unity Toolkit didn't really make sense anymore as it wasn't only for SteamVR so the community decided to rename the project to the Virtual Reality Toolkit or VRTK for short.

The community carried it forward building more and more cool features such as climbing, arm swinging locomotion, different types of grabbing mechanics from using physics to move objects around to simpler techniques such as making the object a child of the controller. But now another problem was emerging, the toolkit had its origins built around how SteamVR was set up and how it worked. The Oculus SDK integration was really just an abstraction layer on top of the inner workings of SteamVR. Harvey realised this was going to cause bigger problems down the line when other headsets and technologies would be released. It couldn't all hang on the underpinnings of SteamVR as this would just be at a fundamental difference to how other headsets in the future could behave.

 VRTK already did so much at this time, and so many people were happily using it to build all sorts of wonderful stuff, but it was clear that it needed to be re-thought, re-imagined and re-built from the ground up in a fundamentally different way that wasn't tied to any particular piece of technology.

Due to the way VRTK had exploded in its popularity meant there wasn’t much time for sense checking decisions or architectural foundations and as it was based in a legacy of supporting SteamVR it meant the codebase just grew and grew around that concept. Kind of like a tumour growing and growing, more code being added by more and more contributors making it harder to maintain and extend. It was so apparent now that VRTK needed to be re-written with some fundamental design considerations at the start.

Rome wasn't built in a day

To re-built VRTK from the ground up was going to be a monumental task, all of what had been built up to this point was done in whatever spare time could be scraped together from busy lives. If VRTK was to really make a go of things it needed funding, but getting investment into something that's free, open source and not based in silicon valley well that was going to be a challenge.

After Valve had declined to offer support due to their strained resources and the release of their own interaction system, other leading industry players had deemed VRTK to be competition to their own open source toolkit offerings so there was also no financial aid to be gained from them. A few private investment opportunities looking to take VRTK and either force it to be moved from the UK to the USA or to make it a private paid product reducing it's accessibility where swiftly considered and rejected. VRTK needed to be for as many people as possible and not tainted by pushing it down the path of a specific corporation's agenda so one potential way of funding it was through crowdsourcing.

The VRTK Kickstarter was launched in January 2017 with the aim to try and gain community support to build the version of VRTK that was really needed to make VR development a thing of ease and beauty for anyone who was interested in doing it or trying it. The reality was that the indie dev community was small, very small and not very lucrative and whilst as many people as possible tried to help out financially, without big name backing VRTK wasn't really going to get the funding it sorely needed. Raising a substantial amount of money through community funding showed a great interest in VRTK but as Kickstarter requires the total amount to be met otherwise none of the funding is received meant that failing to reach the Kickstarter target meant that VRTK wasn't going to be funded this way.

There was a small but vocal backlash about the amount of funding the Kickstarter was requesting, with people thinking it was asking for way too much money and felt it was just a way of Harvey “cashing in” on the existing status of VRTK. The amount being asked for wasn’t for Harvey, it was to fund a small but capable team of people building the solutions for the community and was barely enough to cover the salaries of competent people for 6 months let alone a year of development.

So the Patreon was born, another attempt to try and muster community support into funding VRTK development. Again, the Patreon pledges were generous from the VRTK community but the reality was it wasn't nearly enough to put any serious development into this new re-written version of VRTK. The Patreon had a negative effect too of people thinking VRTK was well funded, whereas the opposite was the case.

Maybe community funding wasn't the solution after all, the community cared deeply about VRTK succeeding but the reality was there wasn't enough that the community could do. Maybe the big industry players just didn't know enough about VRTK to consider it something that was worth supporting? To find out, Harvey took to the conference circuit doing talks about what VRTK offers and networking with as many people as possible to spread the good work that the VRTK community was doing for virtual reality.

The outcome of this wasn't the expected jubilation, in fact it was a cold hard realisation that the industry was filled with corporations hell bent on destroying competition and being the only players in the VR space. How VRTK could be considered competition was a ludicrous thought as it was free, open source and just trying to help move the virtual reality medium forward, but this isn't how it was viewed by many of these leading VR corporations, they saw it as competition that needed stamping out as it took away from their world dominance plans in VR.

This meant that the year of 2017 was rough for the core development team of VRTK, with Harvey putting so much time, effort and own money into the cause of VRTK and getting nowhere with it meant it was likely the new re-write of VRTK would probably never happen.

Many of the core contributors of VRTK had become too busy to help out at the pace VRTK needed it, with more and more new users every day coming into the VRTK community wanting a helping hand to get started in VR development meant the limited time of the seasoned VRTK users was being spent more on helping folks out than actually moving the product forward. The video tutorials were slowing in their release frequency as they were just taking up too much time to produce as well as work on the core code repository whether it was introducing new features or fixing existing issues and bugs.

Another problem now arose around seasoned game developers frowning upon those who used VRTK, it was accused as an asset flipping tool enabling non-game devs to flood stores such as Steam with carbon copy games where the graphics had just been replaced by the free assets downloadable from numerous websites. The “Not Invented Here” Syndrome was also prevalent with many seasoned devs thinking VRTK wasn't suitable for them to use as they could hand craft everything in a much better way, reinventing the wheel and re-solving problems with solutions that were going to be tested for the first time when they released their game. The whole purpose of VRTK was to get people to work together on solving these problems and getting the widest audience possible testing these solutions to work out any kinks, so any time a new way of doing essentially the same thing was built, it was just slowing down the evolutionary process.

All of this negativity was taking its toll on the creator of VRTK, with him barely finding time in his days to spend time on fixing bugs let alone creating much requested features and keeping the community thriving all whilst knowing the current path of VRTK was going to hit critical mass at some point because it sorely needed a re-write.

A few more serious improvements to VRTK v3 were released in the later months of 2017, with some great features such as platform independent avatar hands being largely developed by a long standing community member under the name of JimTheGrim. A few more locomotion techniques and comfort tunnelling which was popularised in experiences such as Google Earth VR, with helpful additions such as providing custom skyboxes courtesy of another community regular JWN.

Windows Mixed Reality support was also built into VRTK by a could of long standing members of the Microsoft Hololens community and support from Microsoft meant that with the free developer headsets provided to the VRTK team it was easy enough to get WMR native support working, meaning any game that worked in VRTK now worked for the Windows Store too with little disruption to developers.

At the end of 2017, the overbearing weight of VRTK not gaining any meaningful support outside of it's own growing indie community meant it wasn't feasible to continue with the development of it. In December, Harvey announced that version 3.3.0 would be the final version he would have time to work on and that VRTK would obviously stay open source and could survive with new features if the community were willing to work on them and agree on their approval.

At this point the VR media seemed to thrive upon the news of the demise of VRTK with click bait-ey news titles and the juice of a good implosion to drive readers to their sites. Where were these media outlets to promote the work of VRTK when it needed the exposure? It certainly was a cut-throat world and something that Harvey wasn't used to or expecting.

The Fourth Episode – A New Hope

There was no period of civil war, no rebels striking their first victory against any evil galactic empire but there was an email, not an outstandingly special email or anything overly exciting. In January 2018, Harvey received an email from the dev team at Oculus saying they had read the news that VRTK was no longer being worked on and they wanted to see what they could do to help. This was nice and all, but not the first time Harvey had receive a “promising” email from one of the main industry players offering to “help” VRTK but they all ended the same way with them wanting to decimate the core premise of VRTK, which was to galvanise a community of people to work on common problems in VR together to evolve the platform by finding common, reusable solutions.

So Oculus had dropped yet another lifeline, was it something to get excited about? Not so much, Harvey had already become bitter to these sorts of offers and had kind of resided to the fact that VRTK, for him, was finished. But Oculus had been nice people in the past to VRTK so it was worth having a chat with them for old times sake. After one phone call with the dev team at Oculus, it was clear they were not messing around, they were serious about doing what they could to support not only the development of VRTK but also the meaning behind it. They were offering a grant to support 6 months worth of development of VRTK v4, the re-write that VRTK sorely needed. The grant wasn't attached with strings of it needs to be an Oculus exclusive or it has to be detrimental to any other VR platform, it was simply good people trying to do good stuff with the resources they have. Like any big company though, things take time to get done, so after a couple of months of working out the fine details it was ready to announce that Oculus were providing funding to VRTK v4 and the news was announced on April 26th, just a few days after VRTK's 2nd birthday!

VRTK v4 was to be a completely new approach to the toolkit experience, rather than pre-built scripts that did a specific thing, it would be fundamental design patterns that could be composed in numerous configurations to provide functionality that was beneficial for VR (or any other use case for that matter). This was so important as it meant whatever changed with the technology in the future, the toolkit would be right there to support it meaning developers could be building with ease for any fledgling hardware that was trying to succeed in the market meaning more games and experiences could support it which would only help with the success of any evolutionary process.

To achieve this lofty aspiration, VRTK couldn't be built by just one dedicated person so the grant funding from Oculus was to be used to bring on a long time contributor to VRTK to work on some of the core features. Christopher-Marcel Böddecker, better known to the VRTK community as bddckr was responsible for many of the SDK abstractions in VRTK making it possible to have one build that works on multiple store fronts and had a deep understanding of much of the core framework. Chris was the ideal choice to be part of the core VRTK team for the version 4 rewrite.

The work on VRTK v4 started at the end of April 2018 with a completely new approach to the way the toolkit would work for developers. Also, due to it's lack of reliance on core Unity3d features meant that the future for the toolkit could and aspired to extend to other platforms such as Unreal Engine, WebVR and Godot to name a few. This premise was even more exciting for the potential of VRTK, if a developer could understand the fundamentals of VRTK and how to create solutions using it in Unity3d then there should be little to stop them from transferring that knowledge to another platform as the only real blocker then is learning the interface of the other platform, but the power to pick and choose the headset technology and engine to build upon would be extremely beneficial to all developers alike.

Teach a person to farm

One of the big passions of the VRTK team and community was making sure VRTK and VR development was accessible to as many people as possible. VRTK is already being used at many hackathons, workshops and educational institutes like high schools, colleges and universities to teach VR development to a new wave of creators. How could VRTK v4 also align the power of the new toolkit with educating those who may be already seasoned developers to those who had no experience at all but wanted to learn? The concept of a VRTK curriculum was devised, was it possible to have a collection of guides, tutorials, videos, learning materials that were helpful to teach the power of VRTK but also in a consistent manner and provide various levels that depending on the user's expertise could have a feasible, understandable entry point?

The ability to bring a whole new medium to a world of new creators is a special opportunity, just like the 80s bringing home computing to the bedroom programmers creating a video game industry, could VRTK help re-ignite such a movement but for VR? The passion to do so is certainly there, with the emphasis on education being on par with actually building the tools to do so. More importantly too having much of the educational content also being free and open source so it can be easily used and contributed to by anyone in the educational space.

Harvey knew that it would be a monumental task to provide top class educational content for VRTK and knew he wasn't best placed to curate such content. One person who Harvey had been following on social media struck a chord with him due to her ability and energy into pushing the benefits of VR. Clorama Dorvilias, also known as CreativeClo, had a great determination in being mostly self taught technologist and pursuing some great uses of VR via DebiasVR to teach the biases in society by utilising virtual reality.

It was going to be a long shot trying to get CreativeClo on board, Harvey had a couple of conversations with her in the past and knew Clo had utilised VRTK for building things in the past. What was there to lose in reaching out and just asking? So Harvey dropped her a quick message and pretty quickly Clo responded with excitement and eagerness to be involved in this educational mission.

Clorama was to head up the educational curriculum with support from the VRTK dev team and work out how best to teach the world about VR and how to build cool stuff for it. Being able to formulate a plan of how to teach a new generation of building for VR was fundamental to the core mission of VRTK so it made perfect sense with the resurgence of VRTK with version 4 to make education a leading priority.

The future is bright, the future is pink

VRTK has been around for a good chunk of time with the advent of the virtual reality resurgence and with the renewed passion and dedication to VRTK v4 it will continue to inspire and provide an essential resource for developers of all levels of experience to create for virtual reality. The community interest in and support of an open source solution to virtual reality problems that isn't driven by the needs of select few corporations has continually and steadily grown over the past couple of years. With a number of these VR leading corporations branding VRTK as competition that needs to be destroyed, it is hoped eventually they will realise that the industry requires collaboration at this level to ensure the success of the medium and not everything has to be about corporate hostilities and there's still money to be made to ensure the ecosystem is lucrative to attract investment and talent to create.

With the future of VRTK bright and hoping to bring in a whole new era of diverse creators into the world of virtual reality can only be a positive outlook for the future of VR. With the premise of the fundamental building blocks of VRTK v4 it can also open the possibilities of using VRTK to build for a whole range of spatial computing such as developing for augmented reality as well. Perhaps another VRTK name change is on the cards for the future as well? After all the new approach to the toolkit with v4 resulted in building blocks not even necessarily being VR-only as the implemented components are generic enough to be used with anything.

The virtual reality medium will only be strengthened by bringing in new and fresh minds from as many different backgrounds as possible. This rich tapestry of creativity will really showcase what the unique virtual worlds can offer us. Doing this together as an open and inviting community that isn't interested in destroying competition is a great way to push forward such an exciting technological advancement.

The future of VRTK is not just to provide a platform for beginners to start their development journey but also to aid and rapidly improve the development process for seasoned developers from indie to AAA houses. Providing reliable, tried and tested tools to prevent them from having to reinvent the wheel means new ideas can be prototyped quickly to find out if their mechanics work or not. The ability to focus on content and not mechanics means that developers can put more effort into producing highly polished content furthering the appetite of the fledgling market but also with the underlying power of VRTK v4 means these developers can further customise and extend the underlying solutions to provide even more unique experiences.

The ability to open up VR accessibility to corporations as well is an important mission for VRTK, to allow industries to quickly and cheaply trial VR solutions to everyday problems means the commercial uptake of virtual reality will be faster meaning more investment will be available for this wonderful new medium to flourish and prosper.

It’s with a heart of hope and love from the VRTK community that VRTK will continue to support the development of virtual reality as a medium and even extend into the future to support other spatial computing sectors such as augmented reality. The future is looking bright and hopefully VRTK is able to make it much brighter.

Comments

  1. Keep up the great work, looking forward to v4!

    ReplyDelete
  2. Great stuff. A great story which ends with a fantastic and hopeful future for VRTK.

    ReplyDelete

Post a Comment

Popular posts from this blog

A new communication channel for VRTK

A new way of keeping up to date with the world of VRTK!