Senior Technical Lighting Artist Josh Marvel shares the story of his career, from filmmaking and digital art in New Hampshire to lighting AAA games in Redmond with 343 Industries. In his decade of work on Halo titles, Josh has helped in-game art technology advance by leaps and bounds, while also finding time to go bounding off into the PNW’s mountains on his skis and bike.
Tobias: Welcome to the Employee Spotlight! Please tell us your name, pronouns, and title
My name is Joshua Marvel, my pronouns are he/him, and my title is Senior Technical Lighting Artist.
Tobias: That's an impressive title! What does your work involve?
I focus on designing, implementing, and testing lighting systems and tools, usually to support new types of Halo experiences that we have never done before. I usually work with artists and engineers – especially with the lighting team and graphics teams. I also create technical lighting content that often makes use of those systems for the first time. The time-of-day lighting (TOD) system is a good example of that.
Tobias: Dynamic TOD lighting was a big new step for Halo. Tell us about how it was implemented into Halo Infinite and your role in that.
For Infinite we chose to create a constantly changing day-night experience that had to support a vast world which was all new for Halo. This involved a lot of research, development, and iteration across many teams with numerous individual contributors over multiple years. The process of designing the whole time-of-day experience and then breaking it down into smaller problems to solve took perseverance, but it was very rewarding. My initial reaction when we first talked about making a time-of-day experience in a large, seamlessly traversable world was: “Wow, this could be cool, but it’s going to be really, really difficult!” I was trying to explain to co-workers just how different this would be from the prior games we had made – both in terms of the design and play experience but also the technology needed to deliver it and get it to perform well. Also, we were supporting multiple platforms for the first time, adding even more complexity! When you make a game you are frequently building optimizations around what the game doesn’t need to do to get it to perform well and leave performance budget to run more advanced graphical features that can make it look better. With the Halo 5 campaign, we had mostly linear gameplay in smaller, highly customized levels. That meant we could precompute a huge amount of information about lighting which saved us performance budget, such as exactly what geometry for sun shadow casting could be removed because the sun had a fixed angle and we could predict that large areas of the world would never cast a sun shadow. This type of heavy precomputation also uses a lot of disk space, and when your world gets much bigger, that doesn’t scale well. Everything about TOD lighting and the huge world blew apart our previous conceptions of how we light the world. For the Slipspace engine, we knew we would have to build a lot of new technology and workflows to make this possible and ultimately create a beautiful game to play.
A small team was formed to create the core time-of-day parameter interpolation system called Dynamic World State (DWS) as well as the content creation tools for it. Most of the environmental lighting in the game is controlled by a series of numerical values that artists use to control the intensity and color of environment lighting such as the sun and sky. Other global systems that affect the appearance and performance of the game such as fog, clouds, sun shadows, reflections and post processing are also controlled by similar parameter values. We needed to define these values differently for each time of day so that, for example, in the middle of the day, the sun shadows appeared crisp with good visibility to the horizon; while at sunset, the landscape is bathed in a warm glow, and a gentle haze covers the landscape. We also needed to interpolate the values between each time of day in very small increments from one frame to the next over the course of many minutes, so that the transition appeared gradual and seamless to the player. I was working in Redmond while most of the engineering work to create this system was done by our partners at SkyBox Labs in Vancouver, and I got a lot of practice in remote collaboration well in advance of the great Covid lockdown of 2020. This team was super enthusiastic to create this experience and contributed ideas as well as engineering support. I specified each lighting element in the game that needed to change during the TOD cycle, which they built into an interface which allowed each time of day to be created individually in a node-based tool, and then dynamically triggered in the engine based on either the time of day advancing naturally, or game events that require a specific time of day to be activated.
I played a consistent role throughout this development process by taking the high level artistic, experience, and gameplay goals for lighting and breaking those down into two parts: features engineers would need to build and pieces of technical content which would be made by myself and other artists on the team. As engineers constructed each new capability in the game, I would test it in iterations to ensure that it was going to look good and perform well, then use that new ability myself to make the world lighting look better across the TOD cycle, while also teaching other lighting artists how to use these new tools to improve other areas of the game. This process of ideating, designing, building, testing, implementing, performance optimization and tuning repeated countless times, and we had to scale it up to encompass several technical artists within the lighting team working with multiple art and engineering teams to deliver the final product.
As soon as we knew that the Zeta Halo gameplay world would be on a series of broken, floating land masses with the iconic ring arcing over you, we started to think through how a TOD cycle could work there. We realized that there is not a consistent horizon like you have on a spherical planet, so the time after sunset would still allow you to see the sun dipping below the neighboring islands. Also, the sun would briefly disappear behind the ring as it passed behind the far side of the ring during the day part of the cycle, creating a ring eclipse of sorts. While this was mind-bending at first, we decided to embrace it and embellish on some of those moments, creating a brief eclipse with saturated colors and streaming light rays through the atmosphere, as well as a nighttime which played off of the bright glow from the sun visible below the floating islands.
If you use the analogy of a symphony to describe how the TOD cycle works, each lighting system in the engine is an instrument that combines with each other to give a layered experience where everything works together in a way that feels natural. DWS is like the conductor’s wand, which controls each system and keeps them all in sync. The campaign art director and myself got to play conductor, telling the sun when to rise, the sky to change color, the fog to burn off the landscape and orchestrating all the other small details that make it work. The lighting and sky part of the cycle are also timed with TOD specific audio and environmental effects. While this does complicate the symphony comparison, it also made the experience much better!
While DWS and these graphical changes were done primarily to make the TOD experience possible, they had the side effect of making many lighting elements much easier to dynamically change in the engine. This improved the workflow and experimentation process for lighting artists and anyone else at the studio who wanted to quickly modify the lighting in the game. We even enabled the narrative artists to animate some of the DWS parameters with keyframes over time too so that they could adjust post processing over the course of a narrative. DWS also opened up a lot of future possibilities for Forge and other Halo experiences, so while it had a core goal, we were able to make something flexible that future things can be built upon.
Tobias: What’s your favorite time of day in the campaign? Least favorite? Favorite place on the map to see your work in action?
There is a time of day called the “blue hour” which is shortly after the sun rises that is particularly special. The angle of the light across the landscape seems to look amazing anywhere. The color contrasts are vivid and the fog drapes across the terrain in a really nice way. We joked that we should just delete the TOD cycle and just have that one because it felt perfect. So it’s my most favorite, but also my least favorite because it was incredibly difficult to optimize in terms of performance! Cascade shadow maps become more costly when there is more geometry in between the camera and the sun. When the sun angle is very low and close to the “horizon” there tends to be a large amount of mountains, trees, rocks and buildings in that line of sight, which made the shadows very costly and way out of budget. We could have just made the sunlight disappear before it got close to the horizon, but it was just an iconic and amazing TOD that neither myself nor the art leads wanted to compromise on that vision. So, we spent a large amount of time finding different ways to optimize shadow rendering and other performance heavy aspects of the TOD cycle just to get these times of day when the sun is rising and setting to look their best without degrading the game’s performance. Watching the sun rise over the distant mountains from any of the high points of the world is still very cool to me today.
Tobias: Outside of time-of-day what were some of your favorite parts of Halo Infinite to work on?
Despite all of the TOD talk, I have created lighting for other parts of the game such as interior campaign levels and multiplayer. On this project, most of that work was to support other lighting artists when it came to using new lighting tools, troubleshooting problems, and optimizing performance. I like working on all parts of the game, but I think I would pick narrative as a favorite because it reminds me of my career before games and in school where I produced short animations with small teams.
Tobias: Can you speak to your ongoing optimization of lighting systems and tools?
Oh yes there is a constantly evolving roadmap of features and optimizations to make Halo better. Sometimes, that is trying to reduce the content creation time so we can create lighting for seasonal content in less time and ultimately aim to get new content out to players more quickly. When releasing a large game, you often don’t have the time to create every piece in the most optimized or perfect way, so sometimes we need to go back and make a particular component of lighting render in fewer milliseconds to either let the player run at a higher frame rate or raise the graphical quality of something else even further. There are also smaller visual details that can always be improved, adding greater definition and realism to the game. A fourth type of improvement is to create new systems and techniques that make richer and more dynamic experiences possible that have not been in a Halo game before, much like the TOD cycle was. Those are some of the most difficult but also the most satisfying things to create.
Tobias: Dynamic lighting is a major driver of graphics improvements in games these days. What do you think makes lighting such a crucial component of a game’s graphics?
When we look out into the real world, all we see is light, and any simulation of a game world that implies a camera or human eyes thus has a huge job in being believable enough for a real human player to get lost in and navigate that three-dimensional world all through a two-dimensional TV or PC display. Since Halo and most other non-VR games are viewed on a flat screen, lighting has a big role to play in creating the illusion of three dimensionality by adding contrast, variation, and depth to the image. The directionality of light sources and shadows is also how the player can tell how all these objects in this 3D world sit in relationship to one another. For example, if we stop rendering a character’s shadow on the ground due to performance limitations, suddenly that character appears to be floating above the ground and it’s hard to tell where they are located in space. That touches on another area lighting affects greatly, which is gameplay. While we want to create a beautiful world full of mood, mystery, and appealing colors that you want to get lost in, you also need to be able to navigate it and shoot enemy characters that might be moving very quickly or are far away and appearing small on the screen. To top this all off, the actual behavior of physical light in the real world is incredibly complex, so simulating it perfectly in real time is not possible. We are always searching for ways to create more realistic lighting within a performance budget that can run in a AAA game that is packed with complex environments, characters, effects, dynamic action, and other things vying for fixed performance and memory budgets. Balancing all these artistic, gameplay, technical and experiential elements while constantly raising the bar on quality and believability keeps me and the teams I work with very busy, and I think that players' appetites for better and more beautiful games have no end in sight.
Tobias: What are some of the most important factors in lighting a game that people may not think about when playing?
As I mentioned, I think it’s easy to take the form and definition that you see in a game environment for granted. Without some careful work, a great environment can look flat, boring or at worst even difficult to understand and navigate. I would also say that we strategize a lot with how we use various lighting tools and systems so that we can deliver something that runs in performance and memory budgets. It’s surprisingly easy to create lighting that looks really great but performs so badly that you basically need to start over.
Tobias: If someone wants to pursue a career like yours, what sort of skills will they need?
It helps to have an interest in the type of things you want to create. As a kid I played a lot of games and was really interested in movies and animation which sort of got me on this path in the first place. I would say the hard skills for making video game lighting are understanding how to make a compelling image with light, and how light works in the real world, which can come from practicing visual arts like painting or photography. Increasingly, new artists just start making game art from the beginning since the tools to do so are increasingly available. Other hard skills that become important as you develop are learning how the lighting you are creating affects other parts of a game, either in terms of how it plays or how it works technically. The more you understand about what it takes to render a light in the scene, the better you will be able to use those lights efficiently.
Tobias: Everyone at 343 has a Halo origin story, tell us yours.
I got interested in computer graphics and digital art quite early when I was in middle school. I lived in a small town in New Hampshire and was a huge nerd for games, comic books, and movies, but the creation of those things seemed foreign and untouchable. I hated to be bored, and always looked for things to study and learn about that could hold my attention. Even then I had a sense that adults needed to work and a lot of them seemed to find their jobs boring. That was a path I really wanted to avoid! I met a family friend who was from my town and had recently moved back after a career as an illustrator for a car company, and as game designer for Parker Brothers. He was actually working remotely in rural New Hampshire as a video game artist on the X-Men game for Sega Genesis in 1992! Right there in his home office he was painting pixel environment art for the game and had a dev kit with a cord running from his PC to the cartridge. I was able to play an in-development version of the game where there was placeholder art, bugs, and crashes. Seeing how a game was created seemed like dark magic. This absolutely blew my young mind as I realized that all these cool things were made by actual people that got to work on them all day! He saw how much I loved graphics and encouraged me to start learning about it on my own. Years later I realized how pivotal meeting this person was in my education and eventually finding this profession. While we hadn’t spoken for years, I actually got an email from him just this last year around the time Halo Infinite was released, and it was very fulfilling to reconnect and share my appreciation for how much he helped me.
In high school, I did some painting, photography, and taught myself computer animation with which I attempted to make a short film as my senior project. Despite a lot of work, I totally failed to complete it, which was a great early learning experience in how to pace myself and plan for a longer-term project. I got the opportunity to attend a college where I could study computer animation and digital art. I lucked into having a professor there who used to work at Pixar and was an inspirational teacher for myself and a number of other students. I worked on some collaboratively-produced short, animated films, and eventually completed one of my own that I spent a year working on, mostly by myself.
After college - in 2005 - I really struggled to find my first job, but eventually started at a tiny animation studio in Western Massachusetts near where I had gone to college. I had done an internship while in school and those connections made this possible. I spent the next 5 years working on a variety of small projects from short animations for several companies including Lego, to advertisements (one of which aired during the Super Bowl) and some pre-rendered game trailers as well. This was a great learning experience where I got to wear a lot of hats and learn a variety of skills. I most enjoyed the final stages of creating materials, lighting, rendering, and compositing for animations. At this point, I hadn’t really worked directly on a game and had mostly imagined myself working on animation - perhaps on feature films one day.
In 2010 my now-wife and I decided to move to Seattle because we loved the city, the Pacific Northwest region, its climate, and the many activities it offers. We had driven all around the country on a giant road trip in 2007 and decided this was the most fascinating place we had seen. I spent a year working remotely for my old job from Seattle - which allowed me to ski a huge amount that first winter – but once summer came, I was pretty desperate to get out of my house and have a newer and more challenging job.
A friend asked me what I wanted to do, and I said: “I don’t know, but I don’t want to work for some big video game company in Redmond because it’s going to be super corporate and not creative work.” Well, I fired up Craigslist that week and quickly found a Lighting Artist contract job for a AAA game, which turned out to be at 343 on Halo 4. When I went to the interview, I found the people to be friendly, and the work something I felt like I could do well even though I had not yet worked on actual real time game graphics yet – only pre-rendered animations. I also loved sci-fi and games, so this was starting to sound pretty fun! I took the job, and a week later I was trying to figure out what builds, bug reports, and all sorts of game-specific tech was.
Tobias: So despite your best efforts, you ended up at the big Redmond video game company. Has the experience been what you expected?
My original guess of what it would be like to work on a big game franchise for a massive company was not right at all! Making a game like Halo is very creative, very challenging, and means you get to work with a large group of some of the most talented and driven people I have ever met. The games are large and complex, so there are many problems to solve and parts of the experience to create. I found it demanded constant creativity, resourcefulness, and innovation. I went from not knowing how to file a bug report in 2011 to having shipped my third Halo game in 2021!
Tobias: What’s been your favorite part of working at 343?
The people at 343 are amazingly skilled, and the culture is very professional with a focus on sharing knowledge. The experience of working here has taught me essentially everything I know about making games. While I’ve been at 343 for a long time, the job has evolved over the years. I frequently get to work with new people both in lighting and other art, engineering, and many other teams, which always keeps things interesting and teaches me constantly. The people at 343 are absolutely my favorite part of working here.
Tobias: Halo Infinite marks your 3rd shipped Halo game, and 10 years in the industry. Do you have any standout memories a decade in?
When I was first at 343 working on Halo 4, all the lighting artists were in a small, dark room in the back of the studio all sitting next to each other. I learned so much at that time, and it was a major struggle to ship that game. We worked really hard and also laughed a lot in that little room. When I look back at that game, I think the lighting holds up well, and I have some fond memories of that time.
Tobias: You picked the PNW to live after road-tripping all around the country. As a Seattle local that warms my cold rainy heart. What’s been the best part about living out here?
It’s really hard to pick. I love skiing and the Cascade mountains in winter were a huge draw, but I’m actually going to say it’s the summers. A nice western WA summer day that’s sunny but not hot or humid is a true heaven on earth.
Tobias: Best place to ski vs best place to mountain bike?
There are a lot of good spots. Alpental for skiing and Tiger Mountain for mountain biking are some of my favorites and popular for a reason – they are amazing, challenging, and not that far away. I’m not sure if I’ve gained more skill in mountain biking, skiing, or video game art since moving here.
Tobias: You mentioned to me that you do backcountry trips into the remote mountains of British Columbia. That sounds awesome.
The Cascades offer a lot of backcountry skiing opportunities, but we are on the doorstep in British Columbia which has an even lower person to mountain ratio, and some of the best backcountry skiing on the planet. Once a year for the last few I’ve headed up there with my wife and skier friends to ski on guided, week-long trips. It’s all human power in that you hike up mountains and ski back down them, but it’s in a remote mountain valley only accessible by helicopter where we stay for the entire week. These trips are easily some of the most fun I’ve ever had – it’s amazing.
Tobias: By the way, congratulations on your new baby girl! I’m about to have my first kid myself. How’s it been balancing parenthood and working on an Xbox flagship franchise?
Thanks, and congratulations to you as well. Our baby was born this November after Infinite was ready to be released but not out yet, so it was quite the experience to reach two major achievements so close together in time. Ironically, it takes a lot longer to make a video game than a new human. Taking the first few months of her life off work made all the difference in the world. My manager and coworkers have been so supportive throughout the process, and I felt really appreciated when I got back to work which eased the transition. Having to balance dad-life with game-making does take some focus, and I need to be more efficient with my time than before. A couple of other 343 people who I’ve worked with for years also had their first babies near the same time that we did, so it’s been fun to trade stories and share that experience.
Tobias: When we first started planning this interview you mentioned swapping work stories with your wife and you had what I thought was a great quote: “Even a bad day making video games is pretty alright.” Got any additional thoughts about the industry to share with folks looking to get into games?
I think it’s different from a lot of jobs because it’s an area with a lot of visibility to the public, and a huge number of fans. This means that a lot of the people who work in the industry are very passionate and worked hard – often for a long time – to develop the skills they needed to make it. That self-selects for people that are very interested in and care about what they do. Occasionally I might take it too seriously, but when things go wrong, I just remind myself that games are for fun, and that we should have some fun making them.
Tobias: Now for a couple of classic Halo questions. Who’s your favorite character in the franchise and why?
I’m feeling pretty mainstream in saying this, but I have to pick Cortana, mostly because I like to see how the portrayal of a central female character in video games has evolved over the years. Also, since she is a hologram and glows, she can be surprisingly hard to light!
Tobias: Favorite Halo storyline?
I enjoyed the Chief/Cortana storyline at the end of Halo 4 because it felt very personal between the two of them.
Tobias: To close out, outside of Halo what games are you playing these days?
During my very minimally available game playing time I’ve been checking out the awesome lighting done by my colleagues at Playground for Forza Horizon 5.