Recently I was honored to be asked to talk about Virtual Reality with a good friend of mine, Andrew Boose. He has been a keen follower of VR and whenever we met he would always have a keen interest in what was going on in the VR space and what was coming next.
His awesome parents of course encouraged this hobby and purchased him his first VR headset, the Quest 2… A great choice if you ask me.
Unlike most kids his age Andrew understands that these devices are more valuable than just playing games and recently asked me to jump on a podcast with him to record some of our thoughts on how these devices are being used outside of the gaming circles,
In this 20 minute podcast we discuss a variety of topics like:
I want to thank Andrew for the interesting discussion and for being interested in such an awesome technology at such a young age. Andrew has a bright future ahead of him!
Recently I have been using the Pico Neo2, a 6DOF device with is comparable to the Oculus Quest and works pretty well for enterprise use cases.
Compared to other headsets it is a lot more open when it comes to working with and managing the operating system, in developing for the headset there were occasions when I needed to share or record the screen to show others what I was seeing.
As with sharing the Quest, previously written about here, Screen Copy (scrcpy) came in very handy with the right configuration for the headsets display.
One thing that is rather annoying about the Pico is that it loves to change screen resolution when you are using it, mainly when accessing what was previously a 2D android based app or settings dialog etc. In VR you wont really notice the difference as they do a great job of handling the 2D apps and giving you an immersive space to work with them but when using scrcpy you often see the output completely change. The only workaround I have found for this is that when a 2D screen appears and messes up the screen display, just run scrcpy with no parameters. This will default to the resolution of the screen and show you the 2D dialog, as soon as you go back into VR mode though you will need to relaunch scrcpy in the below way otherwise you will get a 2 Eye output.
But for most experiences in VR you can use the below instructions to display and record the screen.
Initial Install and Setup
Plug the USB-C cable into your computer and the Pico Neo2.
If its your first time plugging into this computer you will need to accept the prompt within the device that allows you to access the device from the computer and check the box to always allow if you want to skip this section in the future
For Mac the application is available in Homebrew. Install it using the following from a terminal window:
brew install scrcpy
You will also need adb. you can install this by running the following from a terminal window:
brew cask install android-platform-tools
Sharing the Screen connected via cable
Open a command prompt (cmd.exe) or and CD into the extracted folder from above
Run the following command:
scrcpy --crop1280:1776:400:60 -m1600 -b25M
Sharing the Screen connected wirelessly
You will still need a cable to make an initial connection and start the client on the Neo2 and then you can disconnect and use wirelessly, use the following steps to connect wirelessly:
Follow steps 1-4 above in the setup section
Open a command prompt (cmd.exe) and CD into the extracted folder from above
While the device is connected via a cable run the following to find out what your Quests IP address is in the src section:
adb shell ip route
192.168.1.0/24dev wlan0 proto kernel scope link src 192.168.1.198
Now run the following replacing the IP address below (192.168.1.198) as yours:
adb tcpip 5555
adb connect 192.168.1.198:5555
Disconnect the cable from the PC and with the device disconnected run the command to connect to the device as below
scrcpy --crop 1280:1776:400:60 -b 25M
Sharing Window
If everything worked above a new window should appear showing the device screen, you can then maximize this for a better demo window.
Testing was done with variations of the scrcpy command but the above was found to be the best viewing size to enable a good demo.
Recording the screen
There is also a great way to capture the screen while mirroring, add the following command options to the end of the scrcpy command above to do this:
--record file.mp4
To disable mirroring while recording:
--no-display --record file.mp4
Troubleshooting
Connection Issues
If you get a blank window or are unable to connect when device is plugged in you should be able to run the following command from the same folder above:
PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 device
The device should show above rather than:
PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 unauthorized
If you get the latter then make sure you have enabled developer mode and authorized the device as per step 1-3 above.
Target Machine refused connection when in wireless
If you receive a message like the following: “cannot connect to 192.168.1.198:5555: No connection could be made because the target machine actively refused it. (10061)”
Reconnect the Pico to the computer and go from Step 3 in the wireless connection steps above, this is caused because the client on the Pico stopped, normally as you reset the device.
Recently I have been using the Pico G2 4K, a 3DOF device with one controller which works pretty well for enterprise use cases such as 360 video or simple training use cases.
Compared to other headsets it is a lot more open when it comes to working with and managing the operating system, in developing for the headset there were occasions when I needed to share or record the screen to show others what I was seeing.
As with sharing the Quest, previously written about here, Screen Copy (scrcpy) came in very handy with the right configuration for the headsets display.
One thing that is rather annoying about the Pico is that it loves to change screen resolution when you are using it, mainly when accessing what was previously a 2D android based app or settings dialog etc. In VR you wont really notice the difference as they do a great job of handling the 2D apps and giving you an immersive space to work with them but when using scrcpy you often see the output completely change. The only workaround I have found for this is that when a 2D screen appears and messes up the screen display, just run scrcpy with no parameters. This will default to the resolution of the screen and show you the 2D dialog, as soon as you go back into VR mode though you will need to relaunch scrcpy in the below way otherwise you will get a 2 Eye output.
But for most experiences in VR you can use the below instructions to display and record the screen.
Initial Install and Setup
Plug the USB-C cable into your computer and the Pico G2 4K.
If its your first time plugging into this computer you will need to accept the prompt within the device that allows you to access the device from the computer and check the box to always allow if you want to skip this section in the future
For Mac the application is available in Homebrew. Install it using the following from a terminal window:
brew install scrcpy
You will also need adb. you can install this by running the following from a terminal window:
brew cask install android-platform-tools
Sharing the Screen connected via cable
Open a command prompt (cmd.exe) or and CD into the extracted folder from above
Run the following command:
scrcpy --crop1280:1776:400:60 -m1600 -b25M
Sharing the Screen connected wirelessly
You will still need a cable to make an initial connection and start the client on the Pico and then you can disconnect and use wirelessly, use the following steps to connect wirelessly:
Follow steps 1-4 above in the setup section
Open a command prompt (cmd.exe) and CD into the extracted folder from above
While the device is connected via a cable run the following to find out what your Pico’s IP address is in the src section:
adb shell ip route
192.168.1.0/24dev wlan0 proto kernel scope link src 192.168.1.198
Now run the following replacing the IP address below (192.168.1.198) as yours:
adb tcpip 5555
adb connect 192.168.1.198:5555
Disconnect the cable from the PC and with the device disconnected run the command to connect to the device as below
scrcpy --crop 1280:1776:400:60 -b 25M
Sharing Window
If everything worked above a new window should appear showing the device screen, you can then maximize this for a better demo window.
Testing was done with variations of the scrcpy command but the above was found to be the best viewing size to enable a good demo.
Recording the screen
There is also a great way to capture the screen while mirroring, add the following command options to the end of the scrcpy command above to do this:
--record file.mp4
To disable mirroring while recording:
--no-display --record file.mp4
Troubleshooting
Connection Issues
If you get a blank window or are unable to connect when device is plugged in you should be able to run the following command from the same folder above:
PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 device
The device should show above rather than:
PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 unauthorized
If you get the latter then make sure you have enabled developer mode and authorized the device as per step 1-3 above.
Target Machine refused connection when in wireless
If you receive a message like the following: “cannot connect to 192.168.1.198:5555: No connection could be made because the target machine actively refused it. (10061)”
Reconnect the Pico to the computer and go from Step 3 in the wireless connection steps above, this is caused because the client on the Pico stopped, normally as you reset the device.
Recently I was a guest on the VMware Communities Podcast where we spoke about Spatial Computing, Virtual Reality and Augmented reality and where these play in the enterprise, this was a great overview of what my life has been for the last while so I thought I would post it here for those interested in this space and looking to learn more.
You can download/listen to the podcast version here: https://t.co/bpRw8P9fam
Recently I attended a conference where I needed to display a demonstration of what we were doing in VR on the Oculus Quest, the Quest has some great mechanisms for casting to a mobile phone or to a Chromecast device attached to a TV but when it comes to sharing to a laptop there isn’t really a good solution at the moment. Im sure that will all change when they bring out Oculus Link (for a wired connection anyway) but we needed something now.
After digging around on some forums and piecing some data together from youtube, forums and reddit I thought i would make a note of the end result here which enables you to use scrcpy, an android screen sharing utility, to share both with a wire but also wirelessly and also enables an easy way to capture video straight to the computer.
Plug the USB-C cable into your computer and the Quest, this also works with USB to USB-C
If its your first time plugging into this computer you will need to accept the prompt within the device that allows you to access the device from the computer as per below and check the box to always allow if you want to skip this section in the future
For Mac the application is available in Homebrew. Install it using the following from a terminal window:
brew install scrcpy
You will also need adb. you can install this by running the following from a terminal window:
brew cask install android-platform-tools
Sharing the Screen connected via cable
Open a command prompt (cmd.exe) or and CD into the extracted folder from above
Run the following command:
scrcpy -c1200:800:180:320 -m1600 -b25M
Sharing the Screen connected wirelessly
You will still need a cable to make an initial connection and start the client on the Quest and then you can disconnect and use wirelessly, use the following steps to connect wirelessly:
Follow steps 1-4 above in the setup section
Open a command prompt (cmd.exe) and CD into the extracted folder from above
While the device is connected via a cable run the following to find out what your Quests IP address is in the src section:
adb shell ip route
192.168.1.0/24dev wlan0 proto kernel scope link src 192.168.1.198
Now run the following replacing the IP address below (192.168.1.198) as yours:
adb tcpip 5555
adb connect 192.168.1.198:5555
Disconnect the cable from the PC and with the device disconnected run the command to connect to the device as below
scrcpy -c 1200:800:180:320-m 1600-b 25M
Sharing Window
If everything worked above a new window should appear showing the device screen, you can then maximize this for a better demo window.
Testing was done with variations of the scrcpy command but the above was found to be the best viewing size to enable a good demo.
Recording the screen
There is also a great way to capture the screen while mirroring, add the following command options to the end of the scrcpy command above to do this:
--record file.mp4
To disable mirroring while recording:
--no-display --record file.mp4
Troubleshooting
Connection Issues
If you get a blank window or are unable to connect when device is plugged in you should be able to run the following command from the same folder above:
PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 device
The device should show above rather than:
PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 unauthorized
If you get the latter then make sure you have enabled developer mode and authorized the device as per step 1-3 above.
Target Machine refused connection when in wireless
If you receive a message like the following: “cannot connect to 192.168.1.198:5555: No connection could be made because the target machine actively refused it. (10061)”
Reconnect the Quest to the computer and go from Step 3 in the wireless connection steps above, this is caused because the client on the quest stopped, normally as you reset the device.
Recently I took my daughters Softball hitting skills to the next level by putting her up against some of the best pitchers in some of the best stadiums in America…. well kind of anyway!
I joke but this is actually a great use case, imagine putting your softball or baseball college or pro team up against any pitcher that you have recorded stats for and enabling them to hit against that pitcher and any pitch they have made during a game, that’s some next level training there!
We enabled this by attaching a HTC Vive Tracker to the end of an old softball bat she had and using this in conjunction with the HTC Vive VR Headset, this enables the bat to be tracked inside of virtual reality and for a more life-like experience and realistic training method when using some of the softball/baseball games…
Recently I was able to gain access to a Vuze Plus 360 Camera a very niece piece of kit which retails at just less than $1,000, the features are very impressive as you can see below:
360 LIVE preview and LIVE broadcast
4K Stereoscopic 3D 360⁰ Video and Photo
3D audio (spatial audio)
IP65 (water jet proof)
Lightweight and Easy to Use
The Experience
I have always been rather skeptical about the use of 360 pictures, of course there are areas where it makes sense. Real estate for example, this is a great use case for 360 pics and video, being able to look around or walk around a house and see all around you not only on your phone but also in VR can really bring 360 into its element.
Now I have access to a camera however I have to say I am very impressed with the quality and experience it gives to relive a moment in time via picture or video, it really does capture the all round experience and when viewed through VR it gives a sense of being in that place and being able to be in an area and experience it with full perception of the environment and with spatial sound you can also hear the different perspectives as you look around.
Working with 360 Media
Taking a photo with the Vuze is very easy, just hook into their wifi access point on the camera and use the mobile phone software to take a picture or video, couldn’t be easier!
Once you have the picture or video there is then some post processing that needs to take place, the camera tries to do a good job of stitching the individual views together but its not quite there by default. The Vuze come with some great software that make this very easy, it allows you to download the photo and then with the press of a couple of buttons you can adjust the “stitching” and color levels, replace the image at the bottom of the picture (to remove any trace of the camera or tripod) and render the image or video to a given format. Super Easy!
I do however wish there was some kind of bulk mode for this as doing it for 20 or 30 images and videos can get tedious!
Now you have the pictures and Video you need to publish them somewhere or add them to a device, this I think is an area that needs to mature a lot! Its easy to drop the files onto my Oculus Go and Continue reading →
Its been a while since I blogged, mainly because as I focused more on my day job, had a new baby and generally got really busy, I was unable to find the time to share my thoughts on the subjects I was working on, life got in the way!
As of the start of this year however I changed roles at VMware and am now helping to lead the strategy for Cross Reality (XR), Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR) or as the industry has started to call it (and I quite like)… Spatial Computing. Of course I may have dabbled in this area before at VMware ;).
As its easier to blog about what you are working with on a day to day basis I am hoping to start blogging more on these subjects, helping others to learn as I also learn more.
At VMware we are seeing many of our customers already starting to use Spatial Computing to solve certain use cases for the enterprise. Im excited about how this technology will evolve in the future, how VMware will help our customers and what the industry will achieve as a whole.
I of course will keep the history of posts on automation and will continue to keep one eye on whats going on there, maybe even find time to create a few posts when things interest me in this area.
I thank you for continuing to stay interested in the areas I follow and hope you will find this area as interesting as I do, and if you are a VMware customer who has interesting Spatial Computing use cases then don’t be shy, reach out to me and let me know how we can potentially help you!
It gives me great pleasure to announce a new open source project from VMware called VMware VR Datacenter Experience, this is an immersive experience that allows you to put on your virtual reality headset and take yourself into a virtual management environment where you will be able to control your vSphere Environment with the wave of a hand or click of a button.
Details
The VMware VR Datacenter Experience allows you to use a HTC Vive headset to enter a virtual datacenter where a virtual representation of each of your hosts is created by integration with the vSphere 6.5 REST API, once these are created the VM’s are dropped onto these hosts, one for each VM currently registered with the host. In this environment you are able to teleport yourself to each of these hosts and interact with the VMs.
You can pick them up to find out more information about their configuration:
You can even drop them into a trashcan to remove them from your datacenter:
We even extended it to work with VMware Cloud on AWS (VMC) where you could literally throw (vMotion) your VM’s from your On-Premises datacenter into VMC.
Showing you the 2D pictures of this virtual reality environment really does not do it justice, you need to try it to believe it!
Background
At VMware we have internal Hackathon’s where we are able to take a couple of days away from our day job to work on projects which we think are interesting and could provide valuable to VMware, this is a great initiative that fosters
innovation, personally I love working for a company that provides this level of free thinking! Together with William Lam, Yateendra Kulkarni, Vinod Pai and Steve Trefethen, I put a team together and we set out to work on what a VMware Datacenter would look like in Virtual Reality.
I should mention at this point that none of us had developed for VR before, in fact we had never even used the IDE (Unity) before, that was fun! After a couple of days we were pleased when we managed to produce a single host and VMs on that host all by calling the vSphere 6.5 REST API.
After this and some internal marketing and some spare time hacking away at the project we came up with something pretty cool that later allowed me to realize one of my bucket list items and present on the main VMworld Keynote stage with Pat Gelsinger (VMware CEO).
VMworld Keynote
If you didn’t get a chance to watch the VMworld Barcelona Keynote then please do take 5 minutes out of your time to watch the demo below that shows the full remit of what we expanded this project to work with.
Where Can I Get this?
The initial project is now available and ready for you to download, it has been cut down to its basics so we can build on it and ensure the code we are adding and the functionality is based around user input.
And if you want to take a look at the code, ask for more features or even contribute back to the project then please checkout the github repo here: https://github.com/vmware/vr-dc-ex
The Full story
At VMworld I had the chance to catch up with Barry Coombs who interviewed my on this complete story, check out the clip below to learn how a small town boy ended up on stage with a Silicon Valley CEO…
Normally when I create blogs and use the word “virtual” it of course refers to the awesome virtualization technology invented by VMware, this post however is a little different but still just as awesome!
The virtual word that I speak of in this post refers not just to virtual machines but also to virtual reality. Having purchased a HTC Vive earlier this year and being totally inthralled by virtual reality (VR), I started to wonder what an enterprise world would look like if we all had VR headsets, how would we control todays applications in a VR world?
I don’t want to give away too much information just now as the excitement will be at VMworld and on stage, lets just say that managing your virtual infrastructure has never been so much fun, here is a teaser for now:
Want to win a HTC Vive and Alienware laptop?
This year at VMworld I will be presenting an awesome demo on stage with the following members of the VMware executive leadership team and VPs:
works pretty well for enterprise use cases.
works pretty well for enterprise use cases such as 360 video or simple training use cases.
