{"id":5,"date":"2024-11-11T01:40:08","date_gmt":"2024-11-11T01:40:08","guid":{"rendered":"https:\/\/www.visitnyc.store\/blog\/?p=5"},"modified":"2024-11-12T18:08:07","modified_gmt":"2024-11-12T18:08:07","slug":"developing-windows-feature","status":"publish","type":"post","link":"https:\/\/www.visitnyc.store\/blog\/developing-windows-feature\/","title":{"rendered":"Developing “Windows” Feature"},"content":{"rendered":"\n

The idea with the new Windows feature is to have multiple “Windows” using the Window View, but then in them is a RealityView that has a Portal that is sized to be the dimensions of the Window.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n
\n\n\n\n

Turns out streaming more than two videos at 200M (6k per eye) bit rate makes the Apple Vision just stop trying to play the video.<\/p>\n\n\n\n

\n\n\n\n

For comparison, the videos on the Immersive page of the app go through a download process and they are not streamed.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

The downloaded files are 200M-250M bitrate at around 8k per eye. We have found that if the file is over 3GB, the video stops playing or goes into choppy playback. So … a convenient way to fall into an “immersive New York minute” as being a technical constraint found with playing back longer videos. Some of our videos we shot for 2 min 30 seconds each, and for the updated app version with the higher quality videos, they have had to be cut down to a little under 1 minute 30 seconds at max.<\/p>\n\n\n\n

\n\n\n\n

In comparison, the streamed files are stereo 4320×4320 per eye, 50-75MB bit rate at the lowest, and then a few are increased 100-150MB bit rate after rejecting the results for the 50-75MB.<\/p>\n\n\n

<\/div>\n\n\n

Left is 50MB-75MB and right is 100MB-150MB. To be honest, I’m not sure I see a difference or if its in my head. It’s hard to also judge the frame output instead of the playback. I am exporting these via ffmpeg with the two MV-HEVC files that were encoded with a modified sample project.<\/p>\n\n\n\n

\n\n\n\n

Now with the app having high streaming rates, consistent playback requires a fast internet connection.<\/p>\n\n\n\n

I can’t tell if I stuck myself in a NetworkLinkConditioner setting or my Wifi needs to get upgraded, but every time I try to download files on Apple Vision through the Xcode debugger, the network tab always maxes out at 30MB. I checked through my internet connection, my iMac sees 40MB+ speeds for download and I can observe better download speeds on other devices so its not the cloud provider, so I’ve always wondered … is there a max network download speed? Or am I just maxing out my current Wi-Fi hardware.<\/p>\n\n\n\n

Either way, for streaming 3 videos simultaneously, you are going to need a fast connection.<\/p>\n\n\n\n

And then on top, if you are playing three 100-150MB videos, the Apple Vision may start to max out its video playback. Videos in some windows will stop playing all together, and new videos won’t start playing until you close other videos. On top of this, the spatial windows feature has a AVPlayerLooper so there is no break for the video to stop playing. It’s not a great experience, but also very cool to max out the AVP.<\/p>\n\n\n\n

For this, I’ve added this warning:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

I hope users have fun with this feature. While testing this feature during development, I’ve walked around and put our videos on the wall next to each other with other applications windows running and it felt very future.<\/p>\n","protected":false},"excerpt":{"rendered":"

The idea with the new Windows feature is to have multiple “Windows” using the Window View, but then in them is a RealityView that has a Portal that is sized to be the dimensions of the Window. Turns out streaming more than two videos at 200M (6k per eye) bit rate makes the Apple Vision […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-5","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/posts\/5","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/comments?post=5"}],"version-history":[{"count":10,"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/posts\/5\/revisions"}],"predecessor-version":[{"id":150,"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/posts\/5\/revisions\/150"}],"wp:attachment":[{"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/media?parent=5"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/categories?post=5"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.visitnyc.store\/blog\/wp-json\/wp\/v2\/tags?post=5"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}