Tag Archives: Virtual Reality

Screen Sharing and Recording a Oculus Quest with a Windows or Mac Computer

Recently I attended a conference where I needed to display a demonstration of what we were doing in VR on the Oculus Quest, the Quest has some great mechanisms for casting to a mobile phone or to a Chromecast device attached to a TV but when it comes to sharing to a laptop there isn’t really a good solution at the moment.  Im sure that will all change when they bring out Oculus Link (for a wired connection anyway) but we needed something now.

After digging around on some forums and piecing some data together from youtube, forums and reddit I thought i would make a note of the end result here which enables you to use scrcpy, an android screen sharing utility, to share both with a wire but also wirelessly and also enables an easy way to capture video straight to the computer.

Initial Install and Setup

  1. Enable developer mode on your Oculus Quest (https://www.youtube.com/watch?v=1PTFkLnJRLs)
  2. Plug the USB-C cable into your computer and the Quest, this also works with USB to USB-C
  3. If its your first time plugging into this computer you will need to accept the prompt within the device that allows you to access the device from the computer as per below and check the box to always allow if you want to skip this section in the future
  4. For Windows download and unzip the latest build of scrcpy from https://github.com/Genymobile/scrcpy/releases a direct link for the current version when writing this is here: https://github.com/Genymobile/scrcpy/releases/download/v1.10/scrcpy-win64-v1.10.zip
  5. For Mac the application is available in Homebrew. Install it using the following from a terminal window:
    brew install scrcpy
  6. You will also need adb. you can install this by running the following from a terminal window:
    brew cask install android-platform-tools

Sharing the Screen connected via cable

  1. Open a command prompt (cmd.exe) or and CD into the extracted folder from above
  2. Run the following command:
    scrcpy -c 1200:800:180:320 -m 1600 -b 25M

Sharing the Screen connected wirelessly

You will still need a cable to make an initial connection and start the client on the Quest and then you can disconnect and use wirelessly, use the following steps to connect wirelessly:

  1. Follow steps 1-4 above in the setup section
  2. Open a command prompt (cmd.exe) and CD into the extracted folder from above
  3. While the device is connected via a cable run the following to find out what your Quests IP address is in the src section:
    adb shell ip route
    192.168.1.0/24 dev wlan0  proto kernel  scope link  src 192.168.1.198
  4. Now run the following replacing the IP address below (192.168.1.198) as yours:
    adb tcpip 5555
    adb connect 192.168.1.198:5555
  5. Disconnect the cable from the PC and with the device disconnected run the command to connect to the device as below
scrcpy -c 1200:800:180:320 -m 1600 -b 25M

Sharing Window

If everything worked above a new window should appear showing the device screen, you can then maximize this for a better demo window.

Testing was done with variations of the scrcpy command but the above was found to be the best viewing size to enable a good demo.

Recording the screen

There is also a great way to capture the screen while mirroring, add the following command options to the end of the scrcpy command above to do this:

 --record file.mp4

To disable mirroring while recording:

 --no-display --record file.mp4

Troubleshooting

Connection Issues

If you get a blank window or are unable to connect when device is plugged in you should be able to run the following command from the same folder above:

PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 device

The device should show above rather than:

PS C:\Users\alan\Downloads\scrcpy-win64-v1.10> .\adb.exe devices
List of devices attached
1PASH9AF839266 unauthorized

If you get the latter then make sure you have enabled developer mode and authorized the device as per step 1-3 above.

Target Machine refused connection when in wireless

If you receive a message like the following: “cannot connect to 192.168.1.198:5555: No connection could be made because the target machine actively refused it. (10061)”

Reconnect the Quest to the computer and go from Step 3 in the wireless connection steps above, this is caused because the client on the quest stopped, normally as you reset the device.

Fixing HTC Vive Tracker Orientation

Recently I took my daughters Softball hitting skills to the next level by putting her up against some of the best pitchers in some of the best stadiums in America…. well kind of anyway!

I joke but this is actually a great use case, imagine putting your softball or baseball college or pro team up against any pitcher that you have recorded stats for and enabling them to hit against that pitcher and any pitch they have made during a game, that’s some next level training there!

We enabled this by attaching a HTC Vive Tracker to the end of an old softball bat she had and using this in conjunction with the HTC Vive VR Headset, this enables the bat to be tracked inside of virtual reality and for a more life-like experience and realistic training method when using some of the softball/baseball games…

Setting the tracker up was very simple and I soon had the tracker visible in Steam Home.

I downloaded a few games for my daughter to use that I knew supported the HTC Vive tracker:

Pitch-Hit: Baseball

More Info

Continue reading Fixing HTC Vive Tracker Orientation

360 Media & the Vuze Camera

The Equipment

Recently I was able to gain access to a Vuze Plus 360 Camera a very niece piece of kit which retails at just less than $1,000, the features are very impressive as you can see below:

  • 360 LIVE preview and LIVE broadcast 
  • 4K Stereoscopic 3D 360⁰ Video and Photo
  • 3D audio (spatial audio)
  • IP65 (water jet proof)
  • Lightweight and Easy to Use

The Experience

I have always been rather skeptical about the use of 360 pictures, of course there are areas where it makes sense.  Real estate for example, this is a great use case for 360 pics and video, being able to look around or walk around a house and see all around you not only on your phone but also in VR can really bring 360 into its element.

Now I have access to a camera however I have to say I am very impressed with the quality and experience it gives to relive a moment in time via picture or video, it really does capture the all round experience and when viewed through VR it gives a sense of being in that place and being able to be in an area and experience it with full perception of the environment and with spatial sound you can also hear the different perspectives as you look around.

Working with 360 Media

Taking a photo with the Vuze is very easy, just hook into their wifi access point on the camera and use the mobile phone software to take a picture or video, couldn’t be easier!

Once you have the picture or video there is then some post processing that needs to take place, the camera tries to do a good job of stitching the individual views together but its not quite there by default.  The Vuze come with some great software that make this very easy, it allows you to download the photo and then with the press of a couple of buttons you can adjust the “stitching” and color levels, replace the image at the bottom of the picture (to remove any trace of the camera or tripod) and render the image or video to a given format. Super Easy!

Stitching in Vuze

I do however wish there was some kind of bulk mode for this as doing it for 20 or 30 images and videos can get tedious!

Now you have the pictures and Video you need to publish them somewhere or add them to a device, this I think is an area that needs to mature a lot! Its easy to drop the files onto my Oculus Go and Continue reading 360 Media & the Vuze Camera

Slight change of focus.. or should I say reality?!

Its been a while since I blogged, mainly because as I focused more on my day job, had a new baby and generally got really busy, I was unable to find the time to share my thoughts on the subjects I was working on, life got in the way!

As of the start of this year however I changed roles at VMware and am now helping to lead the strategy for Cross Reality (XR), Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR) or as the industry has started to call it (and I quite like)… Spatial Computing. Of course I may have dabbled in this area before at VMware ;).

As its easier to blog about what you are working with on a day to day basis I am hoping to start blogging more on these subjects, helping others to learn as I also learn more.

At VMware we are seeing many of our customers already starting to use Spatial Computing to solve certain use cases for the enterprise. Im excited about how this technology will evolve in the future, how VMware will help our customers and what the industry will achieve as a whole.

I of course will keep the history of posts on automation and will continue to keep one eye on whats going on there, maybe even find time to create a few posts when things interest me in this area.

I thank you for continuing to stay interested in the areas I follow and hope you will find this area as interesting as I do, and if you are a VMware customer who has interesting Spatial Computing use cases then don’t be shy, reach out to me and let me know how we can potentially help you!

 

Announcing VMware VR Datacenter Experience

It gives me great pleasure to announce a new open source project from VMware called VMware VR Datacenter Experience, this is an immersive experience that allows you to put on your virtual reality headset and take yourself into a virtual management environment where you will be able to control your vSphere Environment with the wave of a hand or click of a button.

Details

The VMware VR Datacenter Experience allows you to use a HTC Vive headset to enter a virtual datacenter where a virtual representation of each of your hosts is created by integration with the vSphere 6.5 REST API, once these are created the VM’s are dropped onto these hosts, one for each VM currently registered with the host.  In this environment you are able to teleport yourself to each of these hosts and interact with the VMs.

You can pick them up to find out more information about their configuration:

VM Details

 

 

 

 

 

You can even drop them into a trashcan to remove them from your datacenter:

VM Trash

 

 

 

 

 

 

We even extended it to work with VMware Cloud on AWS (VMC) where you could literally throw (vMotion) your VM’s from your On-Premises datacenter into VMC.

Showing you the 2D pictures of this virtual reality environment really does not do it justice, you need to try it to believe it!

Background

At VMware we have internal Hackathon’s where we are able to take a couple of days away from our day job to work on projects which we think are interesting and could provide valuable to VMware, this is a great initiative that fosters

innovation, personally I love working for a company that provides this level of free thinking!  Together with William LamYateendra KulkarniVinod Pai and Steve Trefethen, I put a team together and we set out to work on what a VMware Datacenter would look like in Virtual Reality.

I should mention at this point that none of us had developed for VR before, in fact we had never even used the IDE (Unity) before, that was fun!  After a couple of days we were pleased when we managed to produce a single host and VMs on that host all by calling the vSphere 6.5 REST API.

After this and some internal marketing and some spare time hacking away at the project we came up with something pretty cool that later allowed me to realize one of my bucket list items and present on the main VMworld Keynote stage with Pat Gelsinger (VMware CEO).

VMworld Keynote

If you didn’t get a chance to watch the VMworld Barcelona Keynote then please do take 5 minutes out of your time to watch the demo below that shows the full remit of what we expanded this project to work with.

Where Can I Get this?

The initial project is now available and ready for you to download, it has been cut down to its basics so we can build on it and ensure the code we are adding and the functionality is based around user input.

Of course you will need the following:

Check out the project landing page here for more details: https://vmware.github.io/vr-dc-ex/

And if you want to take a look at the code, ask for more features or even contribute back to the project then please checkout the github repo here: https://github.com/vmware/vr-dc-ex

The Full story

At VMworld I had the chance to catch up with Barry Coombs who interviewed my on this complete story, check out the clip below to learn how a small town boy ended up on stage with a Silicon Valley CEO…

Virtual Machines meet Virtual Reality at VMworld US 2017

Normally when I create blogs and use the word “virtual” it of course refers to the awesome virtualization technology invented by VMware, this post however is a little different but still just as awesome!

The virtual word that I speak of in this post refers not just to virtual machines but also to virtual reality.  Having purchased a HTC Vive earlier this year and being totally inthralled by virtual reality (VR), I started to wonder what an enterprise world would look like if we all had VR headsets, how would we control todays applications in a VR world?

I don’t want to give away too much information just now as the excitement will be at VMworld and on stage, lets just say that managing your virtual infrastructure has never been so much fun, here is a teaser for now:

Want to win a HTC Vive and Alienware laptop?

This year at VMworld I will be presenting an awesome demo on stage with the following members of the VMware executive leadership team and VPs:

  • Raghu RaghuramChief Operating Officer – Products & Cloud Services
  • Mark LohmeyerVP Products for Cloud Platform Business Unit
  • Guido AppenzellerCTO Cloud & Networking

During this session we will be showing some cool integrations with Virtual Reality and VMware products.

As part of this session we will also be giving away a fully spec’d Dell Alienware laptop and a full HTC Vive headset and accessories.

Believe me, this is an awesome piece of equipment that will blow your mind!

To enter all you need to do is attend the following session, watch the demo and wait for the chance to win this amazing prize!

Session: Simplifying and Accelerating Your Multi-Cloud Strategy [IPC7001KU]

Add it to your session list here: https://my.vmworld.com/scripts/catalog/uscatalog.jsp?search=[IPC7001KU]&showEnrolled=false

Im looking forward to blowing some minds in this session!