The Mixed-Reality Live Video System www.sambiglyon.org
Contents
1. Login SignUp Usemame sambiglon XSplit Applications will have access to Password e View your email address e View your channel s email address and stream key e Update your channel s title game status and other metadata e Cut VODs You are currently signed in to Twitch as vi sambiglyon Cancel This isnt me You will be returned to the Channel Wizard where you will receive confirmation that XSplit has been linked to your Twitch account Click the Next button to progress to the next stage of XSplit setup O Channel Setup Wizard This wizard will help you setup your Twitch channel Please enter your Twitch credentials Account ii The wizard will now search for the optimum Twitch server for your location In the example below the wizard automatically selects the UK as the best choice of Twitch server because that is where the author of this article is located Click the Finish button when you are ready to move on Best target server found EU London UK 7 FPefoming bandwidth test please wait C Channel Setup Wizard Best target server found EU London UK 7 Capable throughput to target server was 790 kbps 7 Based on the choice below the wizard will pre contigure your channel settings to match the desired resolution Your recommended resolution is elias Required Upload Computer Haniware se et EE 48Dp Hip i gt Recommended setting Show2. AE a 4 Any audio that is playing in the virtual environment during the screen capture such as the rising and dipping of the truck engine s noise output as it moves through the gears is also transferred automatically onto the mixed reality video and heard in the real world setting Step 12 But what use is this mixed reality technology if only the person controlling the avatar could see it The excellent news is that thanks to the integration of Twitch video streaming technology everyone with an internet connection and a device with a web browser can see it Going live on Twitch with your production is simplicity itself When you are ready to broadcast to the world all you have to do is go to the Broadcast menu at the top of the XSplit window and click on the Twitch option and whatever you are seeing in the XSplit window is sent to your Twitch channel web page ASplit Broadcaster File View Broadcast Announce Tools elp P Logout sambighon Twitch sambighyon Local Recording Add channel Remove channel C El See ae bee a EN U CENE SOUPCES antl b antl ni Current Scene scene 1 Creative GestureCam Add Remove Settings XSplit locates your Twitch server sends your video stream to your channel in moments and informs you that your transmission is online To stop broadcasting to your channel just left click on the Twitch option again to un tick it File View Broadeast Announce Tool
3. Enter IP Camera into Wikipedia for further details Step Two Each teacher who will be participating in the session should find out in advance what the IP address of their computer is and then give it to the person who will be coordinating the session An excellent website that instantly displays the IP address is What Is My IP http www whatismyip com Step Three The coordinator of the session who is controlling the computer with the XSplit video software installed on it should set up a multi paneled Scene in XSplit as described earlier in the manual but with a minor difference this time In the original version of the XSplit scene setup live video from a webcam was added to the scene construction stage by clicking the Add option to open a sub menu of media types and selecting the Add Camera sub option This time though instead select Add IP Camera from that menu Add camera Add media file Add screen region Add IP camera Add game Add Skype video More sources Step Four A pop up window named Add IP Camera RTSP Stream will appear Enter each session participant s computer IP address one by one by repeatedly using the Add IP Camera option until everybody who will be in the session has been accounted for XSplit Broadcaster File View Broadcast Announce Tools Help P Logout sambighon OK Cancel Scene Sources antl b antl EI Current Scene Scene 1 The central constructi
4. GestureCam Add media file Add screen region Add IP camera Add game Add Skype video More sources Split Broadcaster File View Broadcast Announce Tools P Logout sambighon eatve GestureCam Add Remove Settings Although the camera view is small by default we can stretch it to a larger size that fills the stage by dragging on the corners of the media Before we do so though we will add the second of our two media components to the stage live capture of whatever is on the currently active desktop window Step 8 Return to the Add menu and this time select the Add screen region option A large red cross hairs will appear on the screen k i Le h E O XSplit Broadcaster File View Broadcast Announce Tools Help 365 253 1280 x 768 scene Sources till d till gd Current Scene Creative Gest recam rene Add Remove Settings i SCHEDULE Ce hele MARKETPm The cross hairs is asking us to highlight the section of the screen that we want to capture as a live video stream At this point it is useful to refer to the user interface design of whatever program you are planning on capturing For example if you do not want menus at the top and bottom of an application or menus scroll bars at the side to be part of the video then you can draw about an inch away from the edge of the screen on all sides of it In Second Life though one can hide the user interfac
5. a pre made giant from an online object store such as the Second Life Marketplace APPEARANCE lt Editing Shape Shape o 9 Mouse Male Shape new Height 2 21 Meters Feet Editing Appearance KeGso Sands v Body TF Re EI F a Short A Tall i Y Height Se A Gas Wi TS d 5 b i 2 i i Head EVES Ears Nose Mouth Chin Torso Legs Here is an example image taken during development of this project of a Second Life truck model that was scaled up greatly in Second Life against a totally black backdrop and overlaid via chromakeying onto a real world car park The background removal process is so accurate that you can see every detail of the model in the real world setting in perfect clarity including animated components inside the wheels pure E TETE ETET And here is our enlarged mouse avatar friend again overlaid in 1 1 scale glory on the same car park location Itis incredibly easy to imagine him walking over to one of those cars opening the driver s door and stepping inside This is the immersive power of mixed reality e i ES as ae yi d i Kan vg Ge Us KC GA da J 1 W i ie w a D SEA CSN Waat e Ke sik qf L I EF hp Em 1 I 4 pY RE d i EI r ia ji e 8 IW i in A e Fest ER cf W A7 gl ee EE as 2 Ki i AT d a r KA i DAS o a i P 1 agak j a d 4 KW WA EA d R i E p Tp d K emt A KM K i 4 H Ait A b a a GG a ai j i r F
6. color in use L XSplit Broadcaster File View Broadcast Announce Tools Help Scene Sources antl al i Creative GestureCam Add Remove Chroma Key None Chroma Key Color Key Now that the dropper tool is active moving it to the color on the screen capture preview that we want to make transparent and left clicking on it removes it completely and shows what is underneath Our virtual reality avatar has made it into our world Because XSplit is capturing the virtual world imagery live every single movement that the avatar makes is updated immediately on the merged video feed in the XSplit window It is as though an animated movie is being created on the fly by the avatar s human controller instead of having to craft the movie by hand laboriously and expensively frame by frame over days and weeks He s a bit small though This is not a problem however To enlarge an avatar or object in the virtual world to match the size that they should be in the real world all you have to do is scale up their size in the virtual realm With objects this is accomplished by stretching them with the object editing tools With avatars itis a little more complicated You can resize them upwards to a certain extent with the viewer software s Appearance settings but if a truly huge avatar is needed then it is much easier just to build a giant avatar using prim pieces attached to its body or purchase
7. of mass cooperation that thinks of everyone in the school from administrators through to teachers and the students in terms of individuals who when they come together resemble the cells in the structure of a tree In the structure of a real tree sap rises from the roots up through the trunk and ultimately arrives at the leafy canopy at the peak of the tree This means that nothing living at the top is immune to what is happening at the ground and middle levels Also like a real tree there are both helpful and harmful disruptive elements in the structure of the school Individuals could be regarded as being Tree Cells a play on words of T Cells the cells in animal immune systems with the aim being to ensure that there are many more helpful Tree Cells in the school and that students and staff who are negative cells perhaps because of work stress their family background learning difficulties or other factors in their life are helped to become healthy cells too This philosophy mirrors the saying it takes a village to raise a child with the school community as a whole refusing to turn away and abdicate responsibility for taking care of somebody in that community who is in need of help even if they take some persuading to accept it Each individual in a school can provide help in the areas where they have power to intervene students helping students and making teachers aware of significant problems with friends
8. that are beyond their help and staff helping other staff If there are common problems occurring then they could be addressed with group training programs making use of tools such as the Rift headset and mixed reality presentation technology In the US Military soldiers are paired up as Battle Buddies in a program designed to reduce suicides with each buddy looking out for the well being of their buddy both in battle and outside of it Whilst a single digit minority of soldiers surveyed about the program strongly resented having to be so deeply responsible for someone else s welfare the majority believed it to be an excellent idea Likewise in the internet science fiction comic series Shades of Gray www kahmith com students who enter a military style academy at around 15 years old are assigned a partner who they will work with for the entire five year term of academy life except in exceptional cases where a new partner is assigned due to expulsion of a student or irreconcilable differences between the pairing Success or failure at the academy is dependent upon how well the partners work together if one succeeds then the other does too but the same is true for failure and slipping grades quickly become a powerful motivator to resolve problems in the partnership This formula can be directly related to our real life scenario where the Rift wearer and an adviser on a workstation with Second Life OpenSim installed on it are mutually d
9. to 3 and 5 to 9 at the top of the keyboard Using the number 4 should be avoided as it makes the avatar invisible but there may be productions where this is desired in order to focus on the other objects and avatars in the scene like the director staying firmly behind the camera Any changes made with these shortcuts are non permanent and mistakes can be fixed by closing the viewer down and re launching it This helps one to have the confidence to try out the options assured that if they goof up then a restoration of the normal settings is just a viewer restart away The Advanced menu is a completely legitimate part of Second Life and is documented extensively in the Second Life official wiki website along with a further hidden menu called Develop also documented in the wiki that enables customization of SL to an extreme degree Because the changes that you make to the Second Life environment with the Advanced and Develop menus are localized only to your specific installation of the viewer software on your computer and are not visible to any other users you can configure a virtual film studio that can produce incredible productions for the tiniest fraction of the cost of animating such a production by traditional means Note that some of the settings in the more complex Develop system do persist even after viewer close down though again no other users are affected by the changes Due to the relative user unfriendliness of the
10. to the Oculus Rift via an HDMI cable and connected to the internet via wi fi in order to receive the merged mixed reality video from the Twitch video streaming account AUSB webcam or a digital camera for the adviser s PC that is capable of recording live video A built in webcam in a laptop may perform the same function The camera should be pointed at whatever scene in real life that the adviser wants to be mixed in with the virtual media Optional a second screen to display a copy of the final merged video feed for the adviser controlling the avatar in Second Life OpenSim this video could not be shown on the adviser s main display as that is the one being video captured to provide the virtual reality image extracts An Oculus Rift virtual reality headset currently 350 USD for the Development Kit but projected to be priced lower when the consumer version becomes available Find out more at www oculusvr com The headset is not vital for this project to function as the merged mixed reality virtual and real life video on the Twitch web channel can be viewed on a normal computer display or even a smartphone tablet screen It is however vital for providing the complete user immersion that a virtual reality meta gaming activity requires Second Life or OpenSim viewer software FREE It does not have to be special Oculus Rift versions of the viewers OculusRift Channel for SL or CtrlAltStudio for SL OpenSim This is because the
11. E Add channel Remove channel D THE ADVANCED AND DEVELOP MENUS IN SECOND LIFE If you are a user of Second Life and do not own any land to film productions on there is a way to turn the entire world into a chromakey set The Second Life viewer software contains a series of options in its hidden Advanced menu accessible by using the keyboard shortcut Ctrl Alt D to make it visible at the top of your SL window to turn off SL s graphics elements bit by bit sky water ground etc until the avatar is left floating in a totally black void that is perfect for chromakey removal Advanced Develop Rebake Textures Ctri Alt R Set Ul Size to Default Set Window Size Limit Select Distance Disable Camera Constraints High res Snapshot Quiet Snapshots to Disk Performance Tools Highlighting and Visibility Rendering Types Simple Ctrl Alt Sshitt 1 Rendering Features Use Plugin Read Thread Clear Group Cache Alpha Ctr Alt Shift 2 Tree Ctrl Alt Shift 3 Avatars Ctri Alt Shitt 4 Surface Patch Ctri Alt Shift 5 Sky Ctrl Alt Shift 6 Water Ctrl Alt Shift 7 Ground Ctrl Alt Shitt 6 Volume Ctri Alt Shitt 9 Grass Ctr Alt Shitt 0 Clouds Ctrl Alt Shift Particles Ctrl Alt Shift Bump Ctr Alt Shift Mouse Smoothing shortcuts Show Debug Settings Show Develop Menu Ctri Alt VW a a The above rendering options can be turned off and on quickly by using the shortcut Ctrl Alt Shift and the numeric keys 1
12. S TO PRODUCTIONS Neuro Linguistic Programming is a psychological science that predicts how the mind reacts to specific inputs a smell a sound an image a touch on a certain area of the body etc Once one understands how and why such reactions occur then they can purposely design media to incorporate triggers that automatically activate the impulse in the mind that the designer intended An example is giving students a chapter to read for homework but not asking them to try to remember the contents of the chapter just scan it with the eyes The page contents will be absorbed by the mind and can be drawn instantly back to the student s conscious memory in the next class session by providing them with a media cue that ties in to what they have skimmed during their homework Whilst it may sound sinister in the hands of an educator NLP can be an extremely powerful tool for developing learning programs where students are instinctively and effortlessly absorbing retaining and recalling information NLP is a topic that is too vast in scope to do justice to here in the space available If you would like to read a range of beginner level features about it then an excellent place to start is the Features Archive section of the website of this author s non profit company Sambiglyon www sambiglyon org J FORM A CULTURE OF TREE CELLS IN THE SCHOOL COMMUNITY A system that encourages mass interaction can also be utilized by schools as a model
13. SAMBIGZYON LO The Mixed Reality Live Video System SETUP AND USER GUIDE www sambiglyon org Table of Contents INTRODUC KREE 3 WAR OUNE ED EE 4 GETTING STARTED BUILDING THE MIXED REALITINSNVSTIENM 6 ADPVANCED US Eem 29 A SCALING PROJECTS UP FROM PAIRS TO LARGE CLASS OoROUDS 29 B THEWORLDISYOUR MOVIE SE KE 30 C RECORDING YOUR PRODUCTIONS AS OFFLINE VIDEOS 31 D THE ADVANCED AND DEVELOP MENUSINSECONDUIER cccccccccceeceeneeeeeeees 32 F MERGE REAL LIFE WITH OTHER GAMES AND VIRTUAL WOR DS 33 G CUT AWAY THE WALLS OF THE CLASSROOM TO SHOW A VIRTUAL WORLD BEYOND Ee 34 H USE BACKGROUND CUTAWAYS FOR PROFESSIONAL DEVELOPMENT SESSIONG 36 lmportant Note About Pwitch Terms 0f ServiCe venice eeiseoeccetinie sce eae 43 I ADD NEURO LINGUISTIC PROGRAMMING NLP CUES TO PRODUCTIONR a 46 J FORM A CULTURE OF TREE CELLS IN THE SCHOOL COMMUNTIN 46 EE EE 48 ABOUT ENEE E E E 48 DONATING TOSAMBIOLY EE 48 LEASING AFFORDABLE SECOND LIFE LAND FROM SAMRBlOO VON 49 Guide created by Fleet Goldenberg and Sambiglyon www sambiglyon org INTRODUCTION The non profit educational company Sambiglyon www sambiglyon org of which this author is a co founder has designed and constructed a mixed reality live video setup for Windows and 8 PCs Mac version hopefully coming at a future point that combines an everyday computer and webcam with the hugely popular live web video service Twitch It provides the ability to ex
14. URL i e the other session participants can see the session on a computer tablet or other web equipped device in their home Every movement that they make will be picked up by the IP Camera that is pointed at them and shown live in the video Likewise the camera will pick up whatever they say and automatically incorporate that into the video playing on the Twitch channel too Important Note About Twitch Terms of Service It should be noted that Twitch is not the ideal streaming platform for a professional training session It is capable of handling it but there is a clause in Twitch s Terms of Service that requires that the majority of a broadcast be related to gaming with a small amount of real life footage of broadcasters permitted Whilst use of the system for student activities within a virtual environment is likely to qualify as gaming the same can not be said for a group of teachers sitting discussing Common Core even if they have a virtual world as a backdrop In order to establish a system that can be used time and again without trouble or worry about bans then it is advisable that an alternative video streaming platform to Twitch is used for this project Fortunately XSplit offers the ability to select from a wide choice of alternative web based channels to purchase as expansion modules and stream the finished video production to To access the list open the Broadcast menu and select Add Channel and More Broadcast Ann
15. at that moment I XSplit Broadcaster File View Broadcast Announce Tools Mi Heap Scene Sources ll A atti a Current Seene Screen ieni CH j 1270 x EEN Add Remove Settings Creatve GestureCam Step 11 You may be wondering why we have placed the avatar against a single color background It is because we are going to use a graphics technique from television and movie production called Green Screen also known as chromakey The method popularized by movies such as Who Framed Roger Rabbit but actually existing since the Forties isolates a specific color and renders it totally transparent so that any media laid underneath it our camera feed of the real world in this case shows through and all that will be left remaining of the original image is a cut out of the avatar now a seamless part of the real world scenery Selecting a color in XSplit to be chromakeyed out is very easy First we left click on the text description of the preview panel that we want to perform the chromakey on and click on the Settings option at the base of the XSplit window to open the color settings for that preview panel Alt Broadcaster File View Broadcast Announce Tools Help Scene Sources all 4 atl a Current Scene scene 1 Sareen region 5 27 1275 x 701 dl Creative GestureCam Add k Remove l E ett We want to select black as the color that we cut out from the scene but the only c
16. d support www sambiglyon org SAMBIGL ON LEASING AFFORDABLE SECOND LIFE LAND FROM SAMBIGLYON EDUISLANDS 1 FOR LIVING amp LEARNING The Eduislands are an estate of residential cities in the virtual reality world Second Life that offer land for lease for projects nonprofit education libraries non profit business corporate public authorities personal etc 5 th Our leasing packages start from less than 1 a week We also have a very limited number of waiting list based free starter spaces funded by our non profit activities www sambiglyon org leasing p second Life is a trademark of Linden Research Inc SAMBIGZLYON et co Bb i
17. e Tools Help P Logout sambighon The coordinator should bear in mind that until they go live with the broadcast they will be the only person who can see this view and so there need not be a mad rush at the start of the session to initiate the dropper action You can take your time with it and relax Also remember that XSplit will remember all of the settings changes that you make so you need only dropper a particular IP Camera once and XSplit will store that setting and automatically apply the background cut out effect to that specific participant next time they are taking part in a session Step 9 The final step of the process is to broadcast the merged real life virtual production to the internet so that the session participants can view it on their computer and see themselves together with their colleagues in the virtual meeting space SIT e Announce Tools elp P Logout sambighyon Twitch sambiglyon Yo Local Recording Add channel Rernove channel b A lve view of your Broadcast has been shared A link has been copied to your clipboard Press CTRL V or Right Click and choose Paste in any email editor or chat program to send the link to someone else Clicking on the Broadcast option at the top of the XSplit window and selecting a Twitch video streaming account linked to your XSplit account causes the merged video to be immediately sent to one s Twitch video channel web page so that those who have the channel s
18. e with the keyboard shortcut Ctrl Alt F1 and show the world s visuals in full screen So for our example project in this article we select the entire screen landscape to be captured by XSplit by placing the cross hairs in the top left corner holding down the left mouse button and dragging the cross hairs to the bottom right corner E United 1 OpentgpcpWwrte Dis Edit Yew kret Fan Table Jools W nden Hale kje t aiB eoan Bow E H EE EAE ERATE LOA EE ESE el Default e Timm Kew Aeman i REH Sm St ees A F A z onde Fee oe et ee oe Fe ee ee Hdp Add medis ie Add screen rejin Add F Genen Add pene Add Hype sign ss Faire Source UD jes s T i Ece El gt B E Pee ee When the mouse button is released the X Y coordinates of the screen area that will be captured are set Confirmation of this appears beneath the Add menu and a small preview of the screen capture is added to XSplit s stage alongside the live camera feed Important note when you first see the preview of the screen area capture on the stage it may look weird as though there are multiple copies of the captured image nested endlessly inside one another If this happens to you you can correct the problem by going to the Tools menu at the top of the XSplit window selecting the General Settings sub option and on the resulting configuration window that appears placing a tick in the box labeled Hide from screen region and clicking
19. ependent on each other for success the better that one performs the better that their partner will do in their respective role CREDITS The mixed reality system is designed by Fleet Goldenberg of the UK based non profit company Sambiglyon www sambiglyon org No trademark or copyright claim relating to the system is exercised by Sambiglyon and it may be freely shared constructed and used The text and image content of this guide is copyright of Fleet Goldenberg and Sambiglyon except where otherwise stated but may also be freely shared so long as this credits statement is not removed ABOUT SAMBIGLYON Sambiglyon are a not for profit educational services provider multimedia developer and publisher based in the United Kingdom Company Number 06890732 but offering a range of services that span the globe The name Sambiglyon is derived both from a Greek term meaning Education Entertainment and also our mascot Sam the Big Lion who represents what we do but also our values including friendliness and protectiveness that make us something special DONATING TO SAMBIGLYON The mixed reality system and related information in this manual can be freely used and adapted without permission from Sambiglyon As Sambiglyon is a non profit organization donations made at our website www sambiglyon org are welcomed and deeply appreciated They help us to develop innovative new projects and provide free or low cost products services an
20. ges to overcome We recommend that teachers research on Google and Google Play what the best Oculus compatible way will be to view the video on the Rift via the Android tablet attached to the Rift via HDMI cable using the search term oculus rift video player VR Player would seem to be a good choice of Oculus video player for the Android If you want to view videos on the Rift via a computer the new open source player eleVR http elevr com is also worth looking at So what else could you do with the system Read on ADVANCED USE A SCALING PROJECTS UP FROM PAIRS TO LARGE CLASS GROUPS Any avatar that enters the area of the computer screen that is being captured becomes a cut out and is mixed in seamlessly with the real life video footage in the background This makes it possible for multiple students to simultaneously interact with the student who is wearing the Oculus Rift headset instead of just a two person pairing of the Rift wearer and the computer operating adviser just by visiting the area of Second Life that the adviser s avatar is located at and walking in front of their avatar s camera view Adding additional participants to a session has the potential to be a recipe for chaos though unless all members of the group are working to a common goal For inspiration about how to arrange this we can look to the 1970s where the British boys action comics Warlord and Bullet created a powerful sense of purpose among their y
21. he virtual environment instead and they can act out scenarios without having to awkwardly use traditional control methods Cutaways need not be confined to the classroom either Other areas of the school could also be marked with plain strips that are substituted for digital content such as virtual reality elements or information signage when a live mixed reality video of that location is viewed on a mobile device In schools where every student has a smartphone or tablet it could act as a school guide book that can be easily updated in real time by administrators just by making changes on the computer that is processing and transmitting the XSplit broadcast H USE BACKGROUND CUTAWAYS FOR PROFESSIONAL DEVELOPMENT SESSIONS The above method in Appendix G could also be a useful tool for virtual based professional development sessions Instead of teachers having to learn how to use avatars and digital interfaces or talk through an impersonal interface like Skype they could be given a white sheet or a piece of large white cardboard to place behind them at home and point a cam at All of the teachers in that training session who are doing the same in their homes would then be brought together in a collaborative virtual environment but able to use their real bodies to communicate via the cam that is capturing their voice movements and whatever they are holding and showing to the cam so that the other session members can see it The process for maki
22. in importing media onto the mixing stage in the center of the XSplit window To do so click on the Add option at the base of the window Remove Settings Add camera Add media file Add screen region Add IP camera Add game Add Skype video More sources XSplit offers the ability to import or live capture media from sources such as webcam digital camera static image files computer games and whatever is displayed on the window that the user is currently looking at Other options such as adding Skype video and those listed under More sources are not available on the free version of XSplit and require a subscription upgrade or optional expansion module purchase For the purposes of this article we are aiming to combine a video feed of the real world with extracted imagery from Second Life OpenSim We therefore only need concern ourselves with the Add menu options Add camera and Add screen region Screen region refers to the area of the computer screen that we want to capture and integrate into our video as a live updating image that changes from moment to moment depending on what is happening on the computer desktop Firstly we will add a live image of whatever the camera is looking at onto XSplit s center stage To do this we simply select the Add camera option from the Add menu and click on the name of the camera that is attached to the computer Remove Settings Add camera Creative
23. itled Broadcast Sambiglyon Edit d eu d d A common complaint from users of multiple virtual worlds is the inability to take inventory between worlds The mixed reality system offers a way to address this problem by overlaying objects and avatars from one world onto another environment The integration of living users with virtual tools also provides those users with the ability to attempt to solve the problems that they are dealing with through sandbox experimentation and testing of possible solutions in a way that is not possible when sitting around a table or in a lecture hall in the real world They also do not have to wait as long for a solution as online meetings can be rapidly arranged to suit everybody s schedules instead of having to re organize real life schedules to fit in a special meeting at school or worse wait until the next pre scheduled in school training session There are opportunities to expand such a training program into an open discussion with massed participants from other schools in a district county state within the mixed reality environment sharing best practices with each other and thus filling in specific knowledge gaps at each school The participants in this large scale collaboration could also engage in pro tournament play in the sandbox simulation taking turns to try out different approaches in order to see whose methods work the best I ADD NEURO LINGUISTIC PROGRAMMING NLP CUE
24. logs Redotest C On the final page of the wizard you will be given the opportunity to tweak your Twitch settings There is nothing here that the typical user will have to change so just click the OK button to skip past it and complete setup C Twitch Properties Channel Credentials e Web Authentication Recommended Stream Key Account Sambighyor Deauthorize Tes bandh E Contin ingest server before broadcasting Video Encoding Codec PE Bitrate kbps Mode SS Lef Bud eent Pres refe7 a ft i DOT RUSS Lei att om dei r bat t m l J E See Audio Encoding SE RIDER SUE KHz mono Note Some encoding settings may be disabled for compatibility with Twitch E Automatically save recording on local drive v 1 3 1311 130 Inteleave audio and video in one RTMP channel hi l j E Twitch mmc rk d let i ELT TIL l e fr a L i See am Step 6 You will now be returned to the main screen of XSplit where you can begin to set up the merger of video feeds from Second Life OpenSim via live screen capture and from the real world via the webcam digital camera and blend the two worlds seamlessly together into a single mixed reality view that updates in real time O Split Broadcaster File View Broadcast Announce Tools Help Scene Sources WT b a Current Scene scene 1 Erana 7 ke Add Remove Settings You will observe that there is a panel in the bo
25. ng this possible is more complex than the previous reality merging techniques Requirements for each participant in a session 1 Alarge white sheet or backing board 2 Acam with IP Camera functionality 3 Ameans for each participant to view a video via an internet URL e g a computer or tablet 4 Asession coordinator with a computer that has the XSplit video software installed so that they can receive and merge together each participant s IP Camera video feed and combine it with a virtual environment to produce a final merged mixed reality video with real life and virtual elements 5 Instruct all participants not to wear any clothing or objects that are the same color as the background behind them otherwise those parts of their body will be removed by XSplit on the video Step One Each teacher is sat in front of a white sheet or large white backing board with a cam pointing at them The background should be large enough that the cam s lens cannot see beyond its outer edges The cam being used should be one that has a function in its feature list called IP Camera This means that the cam is capable of transmitting what it sees over the internet live so that another computer that is distantly located from the source of the movie e g not on the same wi fi network can receive that video stream IP Cameras are commonly used for CCTV home surveillance so that the owner of that home can view the it over the internet to check that it is okay
26. olors supported by the Chroma Key option are red green and blue That is no use to us so we need to define a custom color to eliminate instead You may be wondering then why we did not color the board red green or blue One of the reasons for this is that there are different shades of those colors that the chromakey process may not be able to fully remove if the tone does not match up closely with the color settings pre defined in XSplit With pure black though there is no forgive the pun gray area Pure black is always pure black and so selecting that color for chromakeying guarantees that you will achieve an ideal cut out where only the avatars and objects that you want to be visible in the real life footage i e everything that does not contain any hint of black in its textures displays perfectly against the background video footage generated by the camera So instead of using the Chroma Key option we instead define our custom cut out color with the Color Key option directly beneath it Once that option is selected a tool called the Eyedrop Tool becomes available Screen region 5 27 1275x 701 Picture i 7 d Chroma Key None Chroma Key 8 C O Oo r Key Left clicking on the Eyedrop Tool option turns the mouse cursor into a color dropper icon like the one you might see in painting software when picking a specific hue from the surface of an image to make it the current
27. on stage of the XSplit window should now have a series of panels on it each representing a specific participant s IP Camera video feed that will be transmitted across the internet to the XSplit software The panels will not likely have any video playing in them at this point because the session participants IP Cameras will be offline Arrange each IP Camera panel so that they do not overlap one another This will help to ensure that the participants are not obscuring each other on the final merged video Alt Broadcaster File View Broadcast Announce Tool Help P Logout sambighon Scene Sources ill b all EI Current Scene scene 1 Add Remove Settings Add Remove Settings Step Five Add the virtual environment to XSplit by setting that program e g Second Life or OpenSim as the active window on the desktop and bring up the XSplit window on top of it Click on XSplit s Add option select Add Screen Region and use the red cross hairs to set the area of the virtual environment s window that you want to be displayed in XSplit as shown in the example earlier in the manual Senir Lie II BR OR Eck tia red 3163 EE 27 General E Puasa MO F Logout mrbin Th iLL erem Genre Bd rejis de Zaid screen region Jd EP comen id eng Sd Seppe ven Plone SrO The virtual environment in the active window on the computer will be displayed on the construction stage in XSplit Drag its
28. one can find Mac video editing software that offers similar green screen chromakey features to that of XSplit Broadcaster and will work with Twitch For the purposes of this guide we use XSplit on PC because it is very simple to use and proven to work and the author does not have a Mac to test alternative OSX based software packages such as Camtasia For Mac www techsmith com We encourage Mac owners to experiment with their own setup and share their findings with the educational community GETTING STARTED BUILDING THE MIXED REALITY SYSTEM Step 1 Ensure that your USB webcam or digital camera is attached to your computer If you have a built in webcam that you wish to use instead check that it is working Step 2 Sign up for a Twitch account at www twitch tv or log in with an existing Facebook account f Connect with Facebook One click sign in if youve already connected your Facebook account or quickly Sign up for Twitch with Facebook Login Sign Up Birthday Month T Day e Year stile ns Eiar pym pa pl 8 Geatra kt mm that pen La ran nr rrm pe pel y Cicking Sign Up you are indicating that you have d agree to the Terms of Service and Privacy Policy J sign Up Step 3 Visit the XSplit website at www xsplit com to sign up for a free account and then download and install the XSplit Broadcaster software for Windows 7 and 8 from the website The free version of XSplit works very
29. ounce Tools Help D Twitch sambiglyon wl Local Recording Add channel Custom RTMP Remove channel Local Streaming LU Twitch A pop up window with a long scrollable list of online video streaming platforms will appear We recommend that you experiment and find the one that best suits your needs Among the most popular sites on the internet aside from Twitch are YouTube Ustream and Livestream LJ XSplit Broadcaster File View Broadcast Announce Tools a Help P Logout sambighon L Broadcast Plugin Store Justin tv Justin tv is the easiest way to create Ive video and show anyone in the work whats happening right now Nico Nico Douga Nico Nico Douga is a popular video sharing website in Japan Ustream Ustream is the leading liwe interactive broadcast platform YouTube Live The work s largest online video community now offers live broadcasting for YouTubers with more than 100 subscribers This plugin is currenth only available to users with a paid XSplit license Livestream com Be There Livestream is the leading provider for real time event coverage Step 10 The session participants will now be able to follow the live professional training session on their web equipped device at home The session coordinator who has the virtual environment running on their computer can use that world to perform interactive actions display information and drop objects from their in world inventory onto the video Unt
30. oung readers by encouraging them to form secret agent clubs with other kids in their street and their neighborhood Club members built club houses and carried out activities such as charity work exercising to become fitter secret agents going exploring investigating and solving local minor crimes helping people in trouble and giving first aid The comics supported the clubs by publishing accounts of their heroic adventures on the letters page some of which were likely fictional and rewarding contributors with practical prizes like a pennant to hang on their club house wall or a pendant to carry with to inspire courage A teen reader who wore a pendant whilst they tackled an army assault course reported that they felt that their pendant gave them extra strength to climb over 14 foot walls easily Another pendant wearer attributed greater success at school sports to it These instances of heightened performance were likely a form of Neuro Linguistic Programming NLP in which absolute belief in the metaphysical properties of an inanimate object or the values that it represented translated into a greater release of dormant physical and mental potential Another key tenet of the clubs was information security The comics suggested to its young agents that each club use secret passwords and make up its own message encryption code so that their communications could not be cracked if intercepted by non agents usually brothers and sisters who weren
31. panel s corner out so that the panel fills all of the stage L XSplit Broadcaster File View Broadcast Announce Tools Help P Logout sambighon j KRTU ORT Fang KI E TA Step 6 Drag and drop the text descriptions of the IP Camera feeds in the Scene Sources section of the XSplit window to re order the priority assigned to the camera feeds and so to bring them to the foreground as overlays on the virtual world window that is being live captured LJ XSplit Broadcaster File View Broadcast Announce Tools e P Logout sambighon The setup for the teacher training session on the session coordinator s computer is now largely complete The panel configuration is automatically saved and will be ready and waiting for the coordinator the next time that they start up XSplit Before we close down XSplit until we are ready to use it in a real live session though it is a good idea to ensure that the base of the IP Camera panels are positioned right at the base of the construction stage though Doing so means that when the video feeds of the session participants go live their cam feed will not be floating disembodied in the middle of the virtual environment and so break the sense of immersion O XSplit Broadcaster File View Broadcast Announce Tools Help P Logout sambighyon panii P Aey EI ZP Before the day of the professional development session it is a good idea to get at least one of the colleagues who will be at the
32. s P Logout sambighon A lve view of your Broadcast has been shared A link has been copied to your clipboard Press CTRL V or Right Click and choose Paste in any email editor or chat program to send the link to someone else Step 13 Hop over to your Twitch account s personal channel page and after a 30 second delay as you re shown a compulsory advert the price of free services you re a mixed reality world wide web star 7 Untitled Broadcast Sambiglyon Edit Final Step from Twitch to Oculus We haven t forgotten that this article is focused on using this project with an Oculus Rift virtual reality headset As we stated earlier this is entirely optional but once one has the chance to you may never go back to viewing mixed reality on a monitor or mobile Once your live movie is streaming on Twitch the wearer of the Rift headset can view the movie in an Oculus enhanced application such as a movie player observing the merged real and virtual environment in stunning widescreen 3D whilst a student partner and even teachers and other class mates log in to Second Life or OpenSim from computers or mobiles and provide the wearer with virtual augmented information and interactions that helps them to carry out and succeed at important tasks Avatar projections in the headset could also act like a ghost for the wearer to imitate and replicate race car game style a presence that can be helpful or pose challen
33. s pages of the comics Group members could access mission details and logs online and upload collected evidence related to investigations for other team members to sort and process into actionable information and the current Rift wearer to act upon as group leader Emulate the courage boosting pendants with additional internet connected wearables like fitness bands that actually have a scientifically provable and easily measurable effect on the wearer and give them incentive through gamification to live a healthier lifestyle Adapt the password and code creating practices into an educational message for students in the present about taking care of their online safety and security B THE WORLD IS YOUR MOVIE SET Once you have mastered the basics of this system you can attempt much more ambitious productions Instead of a simple board backdrop you could create a huge hollow black cube and use it as a movie studio set to capture anything on the set even multiple avatars and vehicles in motion and chromakey them onto a real life video Bear in mind when designing a production that the avatar of the person controlling the XSplit workstation will be positioned at the front of the screen no matter where the avatar walks to because the Second Life OpenSim camera always follows directly behind it This does not apply to the avatars of others in the virtual environment who are in the local vicinity though if they approach the primary a
34. se options though compared to the Advanced menu anyone using them is likely to be competent at advanced Second Life use anyway F MERGE REAL LIFE WITH OTHER GAMES AND VIRTUAL WORLDS Whilst it would be more difficult to use the chromakey process with games and virtual worlds other than OpenSim and Second Life especially ones that do not support user created content it is by no means impossible All that is required is to find a location that has a relatively uniform texture position the avatar against that surface and use the Color Key tool to isolate a particular color and render it invisible to reveal the real world footage playing behind it Using this approach is not as tidy or predictable than a backdrop with a single consistent color but with persistence you can achieve similar mixed reality results to those in Second Life and OpenSim Here is an example of a good result from extracting the sky color from the sandbox game Scribblenauts Unlimited to substitute it for a real life sky layered beneath BEFORE Scribblenauts Unlimited 5th Cell Media and Warner Bros Interactive Entertainment G CUT AWAY THE WALLS OF THE CLASSROOM TO SHOW A VIRTUAL WORLD BEYOND As well as overlaying virtual media such as avatars and objects on the real world you could also reverse the process and overlay the real world onto the virtual environment An obvious example is to cut out the walls of a classroom in order to view a li
35. session to go live in front of their IP Camera at their home so that the coordinator can test that the system is working Arrange beforehand for that person to have the white sheet backing board background to place behind them so that the background cut out function can be tested Step 7 _ XSplit Broadcaster File View Broadcast Announce Tools elp P Logout sambighon Right click on the text description of that user s IP Camera in the Scene Sources panel to pop up its configuration panel Select the Color Key option and left click on the Eyedrop Tool option that appears to activate the color dropper tool Screen region 5 Si 1275x 70 Move the color dropper tool ringed in the image below to the white background and left click on it to select the white color as the shade that will be removed by the XSplit software revealing the virtual environment behind the image and leaving the colleague as an impressive looking integral part of the environment Aalt Broadcaster j File View Broadcast Announce P Logout sambighon ER Step 8 On the day of the professional development session once all of the session participants IP Camera feeds are active on the XSplit screen the coordinator can quickly repeat the color dropper action in the configuration settings of the other cameras to remove the white background from those colleagues as well Split Broadcaster i File View Broadcast Announc
36. student controlling the virtual environment who is partnered with the Rift wearer will not be wearing the Oculus headset but will instead be focused on generating the avatar actions object manipulations and activities that will be combined in real time with the real life camera footage to create what the Rift wearer is seeing on its internal display A Twitch web video streaming account FREE Sign up at www twitch tv Important note Twitch s Terms of Service TOS require that the majority of a broadcast transmitted via its service features content related to gaming virtual worlds with a minimal amount of footage of human presenters If you anticipate that this would not be the case for your mixed reality project for example if you are going to use it for teacher professional development sessions with long running webcam sessions with real people instead of digital avatars then we recommend using an alternate online streaming platform that does not have this restriction See the Important Note About Twitch Terms of Service section of this manual s contents page to read further information about selecting an alternate streaming platform that is compatible with the mixed reality system An account with Twitch s recommended PC video editing software XSplit Broadcaster for Windows 7 and 8 FREE Download the software and register an account at www xsplit com This system may be able to be used with Mac computers if
37. t in the club Clubs also often found their own solutions for problems that the comic editors never anticipated Members who reached their mid teens and felt that they were too old and grown up for their club and should quit instead became leaders of the club and acted as the boss who coordinated activities and arranged missions for the younger members Some clubs even charged a small weekly fee that was pooled to purchase equipment and uniform for members such as compasses first ad materials and wet weather boots It is no longer the Seventies though and society has changed greatly for the worse in some respects Kids no longer care about health as much as they used to and the risk of child endangerment in public places is higher or perhaps just more commonly reported in the news So whilst it is not realistic to closely emulate that era there are lessons provided by it that can be adapted for the present day Schools can use these insights to expand an Oculus empowered program where only one student at a time can use the Rift unless the school can afford more than one headset and attached tablet into a project that involves the entire class in a way that means they are excitedly assisting wnomever is leading the group at the time by taking their turn to wear it Some examples include the following Documenting students progress on a blog or social network similar to how clubs reported their news to each other via the letter
38. tage If this is not the case the under and over position of the two previews can be switched just by holding the left mouse button down on the text description of one of the previews in the bottom left Screen Sources panel of XSplit and then dragging and dropping that highlighted preview on top of the text description of the other preview in that panel Individual preview feeds on the stage can also be turned on and off at any time by ticking and unticking the tick box beside the descriptions Step 10 With our camera feed and screen feed set up we are now ready for the really fun part capturing live moving imagery from Second Life or OpenSim on our XSplit stage Start up your SL or OpenSim viewer software A normal non Oculus viewer will suffice at this setup stage of the project Find a location in world where you have object construction or rezzing powers This may be a piece of land that you own or a public sandbox area of the virtual world where anybody can freely build Create a basic cube shape stretch and flatten it into a large square board and change its color to black Position your SL OpenSim avatar against the black backing board you have created and bring up the XSplit window so that it is overlaid on top of the viewer window This makes the viewer software the screen image that XSplit is capturing and the screen capture preview on its stage changes to a live copy of what is displayed in the viewer window
39. the Apply button ASplit Broadcaster File View Broadcast Announce Tools W Help Logout sambighon Scene Sa wll d all si Current Green region 5 4 1270 x 760 Creative GestureCam Add Remove Settings If we now look at a different desktop window then XSplit instantly recognizes this and updates the preview on the stage Step 9 The two video feeds are no use to us though at their present size We want to expand them so that both of them completely fill the space on XSplit s stage overlapping one another perfectly To do this place the mouse cursor on a corner of the preview to make the traditional double arrowed window resizing icon appear and then hold the left mouse button down and drag the corner of the preview until it occupies all of the stage space Repeat the procedure for the second panel as well O Split Broadcaster File View Broadcast Announce Tools Scene Sources WT b WT d manor E Screen region 5 4 1270 x 760 H Creative GestureCam Add Remove Settings O Split Broadcaster File View Broadcast Announce Tools we Creatve GestureCam Remove Settings Help ae There is no need to worry though We are going to cut out part of the scenery of the screen capture panel later so that the preview containing the live camera footage becomes visible behind it The preview containing the screen capture should be on top of the one with the live foo
40. tract avatars objects and other digital content from virtual reality worlds such as Second Life and OpenSim as cut outs via a graphics technique known as chroma keying or green screen and overlay them seamlessly on live video footage of the real world as though they were a PNG image The combined video output can be seen via a live updating Twitch channel page viewed with an Oculus Rift headset or on a web browser page and even recorded as a permanent FLV or MP4 video file Best of all aside from the cost of a tablet and an Oculus Rift headset and even the headset is optional with this mixed reality system depending on how schools plan to utilize it the grand total price of setting up your own version is zero The two online services that provide the technology to make it function Twitch and XSplit are free and without time limit It is said that seeing is believing So here as proof is just one example of the kind of end result that can be achieved showing the live video output of a Second Life avatar merged with a real world car park on a Twitch channel page Untitled Broadcast sambiglyon Edit el m r A WHAT YOU NEED Below is a full list of what is required for this project APC desktop or laptop for a student in the role of an adviser who will control the media that is being created and mixed together An Android tablet for the Oculus headset wearer to carry that will be attached
41. ttom right corner of the XSplit window with a series of Scene buttons in What this means is that you can configure multiple video mixing setups with different media sources in each and switch between them at will at the touch of a button This is especially useful for producers administrating a live broadcast where they need to rapidly switch between different video streams and graphics It also enables teachers to set up a wide range of configurations for their classes well in advance instead of wasting class time fiddling with settings For the moment though we do not require any of that complexity and so we can leave the panel set on the default Scene 1 and ignore that part of the screen from here on If you need to enlarge the XSplit window before you begin putting together your mixed reality production you can do so by going to the View menu option selecting the Resolution sub option and picking a resolution that is comfortable for you Alt Broadcaster File WW Broadcast Announce Tools Bai Help k Logout sambighon Resolution 426 x 240 240p 16 9 Frame rate 640 x 360 360p 16 9 Transition 768 x 432 432p 16 9 Transition speed 00 ms 854 x 480 480p 16 9 Projector 960 x 540 540p 16 9 Hide viewport 1280 x 720 720p 16 9 Scale viewport Edit resolutions Scene Sources till d till E Current Scene cen 1 Scene 10 Scene 3 Scene Add Remove Settings Step 7 We are now ready to beg
42. vatar from a distance then they will be initially visible as small shapes in the background of the real life video footage and steadily grow larger as they walk towards the front of the screen in the same way that an approaching person in the real world would be perceived C RECORDING YOUR PRODUCTIONS AS OFFLINE VIDEOS Speaking of movie making XSplit isn t just for live streaming you can record your live mixed reality productions as permanent offline movie files as they happen Starting a recording is as easy as starting a broadcast Go to the Broadcasting menu and click on the Local Recording option Your movie will then begin automatically recording to a folder on your computer as a compact sized FLV format video that can be uploaded in minutes to YouTube Starting and stopping recording will not interrupt your live broadcast to your Twitch channel To stop recording click on Local Recording again to un tick it You can find your recordings quickly by going to the Tools menu option at the top of the XSplit window and clicking on My Recordings to open the folder where your FLV files are stored The paid subscription Personal and Premium accounts for XSplit have the significant advantage for professional video producers of also allowing recording of videos in the high definition MP4 video format as well as the standard FLV format Solit Broadcaster View Broadcast Announce Tools cy Twitch sambiglyon Th RRC R
43. ve controllable virtual world behind them i e laid underneath the camera feed of the classroom in the XSplit software This can be accomplished easily just by covering the classroom walls with fabric sheets such as curtains that are of a single plain color throughout Selecting this color with the dropper in XSplit will cut away the curtains and display the virtual environment laid beneath the camera feed Using this method the classroom and its students and equipment can be transported anywhere from outer space to a Clearing in a beautiful forest And because the virtual environment is displaying in real time it can be controlled by a student or teacher If the students were controlling their avatars at a classroom computer then they could even watch those avatars interacting with the world behind their walls in life size responding to the actions that they input into their computers Controlling those avatars hands free via motion sensing devices such as Kinect for Windows and the Creative Gesture Camera instead of a mouse and keyboard or joypad would result in an even greater sense of immersion This could also be an easy way for younger students to make games using the cut away classroom as a new form of video game development studio that focuses primarily on creating and performing outstanding stories Instead of having to learn drawing and coding their real world bodies hands and equipment could be used in conjunction with t
44. well and has no time limit though upgrading to a paid account from just 14 95 for a 3 month subscription period provides many additional benefits such as unlimited frames per second streaming speed recording videos in MP4 format no watermark logo displayed on videos and capture of video footage from Skype Step 4 Start up the XSplit Broadcaster software and enter your XSplit account details into the pop up login box that appears XSplit Split Broadcaster Email On the next page is an ad for the paid versions of XSplit with a 10 second timer that counts down to zero before you can continue When the timer reaches 0 click the continue button to complete the login process Step 5 Go to the Broadcast menu at the top of the XSplit window and select the Twitch option This will cause a Channel Wizard configuration interface to open The first step of this setup wizard is to authorize XSplit to access your Twitch account Click on the Authorize button to begin the account linking process O Channel Setup Wizard This wizard will help you setup your Twitch channel Please enter your Twitch credentials lel i thes liebe a rss aan LA m m m m y Click Fut Th nz to lir In OUP Th men GOU it Enter your Twitch username and password and click the Login button then the Authorize button to complete the account linking ASplit Broadcast otf Authorize XSp it Applications to use your account
Download Pdf Manuals
Related Search