Pipeline Guide intentions
Make this into an article?
Maybe this post could be made an introduction/intention/explanation article for the Immersive Pipeline Integration Guide?!?
Kartaverse Workflows | Immersive Pipeline Integration Guide 512 pages https://docs.google.com/document/d/1tewIaHZh8mWI8x5BzlpZBkF8eXhK2b_XhTWiU_93HBA/edit?usp=sharing
A recent question on LinkedIn asked me to summarize what the Kartaverse Pipeline Guide was about: Q.
So this giant document is an overview of the Kartaverse? And it is platform agnostic? More theoretical?
My response on the effort: A.
I try to take a wholistic view on pipelines that focuses on the task being achieved and what tools/resources are already on-site. My efforts are more on getting people to think about automation and efficiencies that can be had. Aka don't do media management on hundreds or thousands of assets by hand, automate backups, speed up slap-comp creation, etc... The process of selecting which tool to use these days seems to matter less and less compared to asking if the whole workflow is supporting the creative process or acting as a boat anchor. I spend so much time on automation and scripting to try and multiply the effective volume of work that a tiny self-funded team can achieve. If you skim the Google Docs sidebar category for the guide you will see it covers AWS deadline usage including using MistikaVR as a deadline job, how to install and use Resolve + Fusion + Fusion Render node. It has topics about Python and Lua scripting including a discussion of QT window UX creation. A bare metal Windows 11 step by step install guide including environment variables, and GPU performance tuning. A step by step coverage of bare metal install of Rocky Linux 8.5 including GPU driver config and all common VFX apps on Linux including Maya, Blender, V-Ray Benchmark, V-Ray Universal batch render, and Pixar's RenderMan + Tractor. Also s bare metal install guide for CentOS 7.9 + Cinnamon Window manager, GPU drivers and common tools. Also common RAW image processing, panoramic stitching workflows with PTGui Pro are discussed, along with a dissection of the PTGui Pro v11-12 json based .pts format and a listing of how to access all relevant controls via a programmatic fashion. A step-by-step explainer of ST Map based panoramic template generation for live 360VR and offline panoramic stitching is mentioned. Including notes about TouchDesigner based real-time stitching via STmaps with live capture from SDI/HDMI digitizers and network feeds like NDI. Also the same STMap approach is explored in Resolve/Fusion. A breakdown of common SfM based image based modelling tools is listed. Along with important notes about how to get Reality Capture able to be bent to the will of people trying to do 4D volumetric video mesh sequence generation through the use of image sequence based multi-view workflows. And tips like re-using exported Reality Capture XMP formatted 3D camera locators, XMP based bounding box region cropping effects, and the actual Reality Capture export options themself. The Agisoft metashape SfM image based modelling toolset is demonstrated with a command prompt/shell based code demo on how to set up a local volumetric video render cluster with client/server bindings for a small farm. RAW media processing and HDRI exposure brackt blending workflows are discussed including notes on how to Automate Photomatix to generate proper HDRI EXR files. A breakdown of the former Nikon camera RAW tools which are now sold as DxO Optics PhotoLab is given with a step-by-step explainer on how to access the 100% undocumented hidden RAW processing command-line interface. A listing of common open-source workflow tools used in media & entertainment post-production is given. A breakdown of the core concepts for OpenColorIO based ACES color managed workflows is included along with a summary of ACEScg and ACES 2065-1 deliver options and a comparison against linear gamma 1.0 color managed workflows and common Rec.709/sRGB options of the past. A coverage of the rescuezilla open-source bare metal Win/Linux disaster recovery, and disk to disk full-drive imaging cloning/backup tool is included. Tips explain how to use the gParted and Disks drive utilties on Linux to shrink a Windows NTFS or Linux based complex filesystem drive partition. Notes explain ways to solve a drive layout problem that stops a drive to drive clone from working from a larger drive (with lots of empty space) onto a smaller drive that is big enough to hold the active data. Common LTO drive backup tools like Canister Hedge are discussed. The USB thumbdrive ISO/boot media cloning tool called Balena etcher is discussed to show how to prepare boot media in the case a low level OS reload is needed. Discussion of common network utilities is shown. Common web development practices like setting up a local content staging server to allow you to prototype a media heavy website creation task without needing to constantly re-upload footage via the open internet to an external website. A demonstration of how to get Meta Quest HMD drivers installed, including the use of AirLink/Quest Link to allow a connected PC workstation to use a self-standing Quest HMD as a review tool. EXIFtool and other tools are shown for how to work with movie and image metadata. A discussion of nodal workflows including "data node" concepts for automation are displayed. A step by step guide on how to get Houdini and Houdini's bundled license-free render manager tool HQueue is presented. Houdini's node based "render manager" like system called PilotPDG/TOPs (task operators) is demoed and a side presetation shows how to make custom nodes in TOPs to arbitrary tasks like controlling the creation of a full composite in an external VFX tool, and using TOPs to also manage the distributed rendering operations. Houdini centric Pixar OpenUSD approaches are examined including way to streamline TOPs (task operators) to perform Alembic and USD ROPs (render operations). There is a side topic on how to set up a full Windows build environment including Git, Visual Studio, NVIDIA OPTIX, CUDA Toolkit, CMAKE, and other libraries like NVIDIA InstantNGP so neural graphics primitives based NeRF scene reconstructions can be created and explored interactively. A step by step workflow for applying "Ai" based image generator techniques that use the OpenAI "Dall-E" prompt based image generator toolset by hand from the command prompt is demoed, including setting up an OpenAI free account from scratch. Then a data-node presentation shows how arbitrary comp-specific tasks (like doing Dall-E AI image generation) can be done in a code-less fashion to allow any comp to synthesize new novel ML based visuals with 4D timeline based variations shown. A coverage of OpenEXR Multi-channel and multi-part media I/O is given, along with a comparison of encoding said multi-view EXR media between flat image sequences, tiled texture atlas and EXR multi-part/multi-channel representations. A discussion of truly parallel I/O based multi-view data processing techniques is given. Houdini Tokens work based upon environment variables. The tokens can be used in filenames for loading/saving content in a Houdini node graph are explored. Environment variables on macOS and windows are explored, including notes about macOS Launch Agents vs Terminal window defined environment variable scopes. A side-companion guide shows how to "reverse" the YouTube360 cubic format media you rip from their site back into spherical mono 2D or Stereo 3D formats. Plus a full breakdown of Resolve Studio, Resolve (Free), Fusion Studio, and Fusion Render Node is given. An introduction to Resolve Fusion page/Fusion Studio centric nodal workflows and a UI overview is given for first timers who have never used the toolset. A detailed coverage of all ways Resolve and Fusion can be customized via custom scripts/macros/templates/hotkey bindings/menus/OFX Plugins/Fusion native plugins/a fuses is given. And extensive coverage of common 360VR footage transformation operations is shown. And a novel Mobius transform tool that allows XYZ translation effects to be carried out on a classical 2D mono or stereo 3D pano. And coverage of 360VR and normal camera extraction workflows to be able to apply camera native recorded IMU accelerometer/gyro data in a node based fashion to do smoothing of hand-held motion. This is done with a thing called data nodes and an open source tool called Gyroflow. Did I mention I love workflows. LOL Now I'm more focused on making a viable alternative to Unreal Engine 5.1 + nDisplay + Disguise that is a performant interactive ML (machine learning) driven multi-view display generator for fulldome, projection mapping, 360VR, passive stereo displays, and virtual production usage. What IMHO sets my humble dev effort that is a WIP apart from conventional VP techniques by Disguise is my effort to apply true volumetric photoreal NeRF rendering to the task of making large scale volumetric representations of live-action captured locations. I call this concept nVP (Neural Volumetric Production) and it relies on building a feature rich tech stack ontop of NVIDIA's latest and greatest InstantNGP NeRF library that was just released this year. And I'm working to make the nVP workflows Pixar OpenUSD based so camera 3D locators and image planes can migrate from conventional DCC packages, into SfM image based modelling tools, over to common NeRF tools. That's most of what the pipeline integration guide content is about.