Introductions and Daniel’s background and experience

Pete (00:11):
Alright, well welcome to Spotlight on. I'm Pete Clark, your host. I'm here with Dan Sturman, who is the CTO of Roblox. So a couple of things: Dan, who are you? And more importantly, what is Roblox?

Daniel (00:23):
Well, I'm thrilled to be here with you today, Pete. Just a little bit about myself. I've had a fairly unusual path to where I am at Roblox. I started my career as a researcher at IBM, and did a number of jobs there over the course of a decade. I moved on to Google where I worked on large-scale systems. Then joined Cloudera, which is where you and I originally met, where I was the head of engineering there and Cloudera, in case folks aren't aware, built big data software, kind of coined the term big data at the time, and now I'm at Roblox. And Roblox just looks like a very, very different take. But it was for me, just bringing all these things together. What Roblox is, it's an immersive 3-D platform that is social for people to build 3-D experiences. So you could think things like games, you can think concerts, you can think now brand experiences, anything where you're trying to model the world.

(01:21): The term that, of course, everyone has floated around this is metaverse, but we think that's a very broad and not very specific and kind of confusing term. So we just say what it is, which is 3-E immersive, interactive, experiences. It's brought together all the stuff I've been doing in my career from very systems-oriented because we are a large-scale infrastructure provider – it's how all these experiences run – to a consumer focus and building development tools for our creators to now bring in incredible technology around AI. And of course, underneath all this is graphics. And that was part of the attraction of my joining was I got to go do something completely new and the graphics part was absolutely that, right? So it's a really broad-based platform where we have a community of users who come and enjoy experiences, but most importantly a community of creators who are the ones who bring everything to the platform. Roblox itself does not create any content.

(02:17): We just provide the platform and this incredible community of creators come build things out and users come and enjoy and interact with each other. Everything we do, and I think this is important, is in service of enabling our users, our creators, to be more successful on the platform. It's a fantastic place to be. And we've been doing a lot of cool stuff over the past four years.

Pete (02:39):
It's kind of a full-circle moment for us with Spotlight On. So when we had the idea for creating some content, sort of a blog post that we dubbed Spotlight On, and we thought we'd interview great leaders around different topics, and Dan, you were the first person that we interviewed.

Daniel (02:57):
The number of times people come back to me with questions about what we did there, because that was all about hiring, if you recall. And so I think it was one of the best things ever. People seem to really appreciate it and learn from it.

How AI is changing safety and civility on the Roblox platform

Pete (03:13):
Yeah, for sure. And we still point people to that. So shameless plug, the blog post is still up. It was a three-part series and it's just some timeless content. Let’s start a little bit with just sort of how you've thought about it at Roblox, just the impact of AI, how you looked at just where this is going, what it means to the business, and where you've leveraged it.

Daniel (03:35):
First of all, Roblox has been doing AI in one form or another for a while and in some cases in the very obvious places. So for example, collaborative filtering on which of the millions of experiences we recommend to any given user. But an area that's really been deep has been in how we think about safety and civility on the platform.

(03:53): You really just cannot do that at scale without a lot of automation assist on whether it's text and text filtering or content understanding. That's been central to our approach there where we use a mix of human-based moderation but strongly assisted by AI. But what's happened in the past, I guess maybe it's been 18 months, two years, right? An explosion with very large models and the capabilities there, I think are something that Roblox is very well positioned to take advantage of and we're leaning in. The two main places, where that is impacting us is with first of all, helping creators create faster, better, higher fidelity. And then the second big one continues to be in the safety area. The ability of these models to better understand content of all types allows us to do new things that maybe safety concerns would've prevented us from doing before and do them well and do them at scale. So it's a very exciting journey for us on both those fronts and we've done some stuff I think is pretty cool and innovative and maybe we can talk about that a little bit.

Pete (05:07):
Yeah, well maybe even on the trust and safety side, just I think when you get into the world of user-generated content or allowing creators on a platform, obviously trying to give them the freedom to express what they want to, but then at the same time make sure it's a safe place for everyone. So how do you think about that just in terms of any unique aspects to some of the technology or tools you're using for that?

Daniel (05:32):
Yeah, well I think first it starts with our approach to safety, which I think is a little bit different than some companies because Roblox started as a platform for kids. We've always had in our core this belief that safety and civility, age-appropriate safety and civility was incredibly important, and was a must-do with any product release.

(05:52): So then that leads you to, you're building a great product, you have a new capability, you have to be designing the safety systems hand in hand. And I'll give you a great example of this on something we just launched that I'm very excited about. For probably about a year, year and a half, we've been looking at how do we bring real-time voice onto the platform rather than just typing messages and doing text. We realized users wanted to be able to talk to each other on platform and because it's Roblox and it's 3-D, that voice should be spatial. So I can whisper in your ear and someone else might not hear it, but I can shout more focus. It has to be real to the 3-D world. But when you do all that, you'll think about how are you going to moderate this?

(06:32): And about last December we were looking at this problem and saying, look, the original approach we thought we had, which would be mostly human leaning with cues from users, was just not working with our pilots. It wasn't giving us the level of civility we wanted on the platform. And this is a 13+ feature by the way. So even for that audience, this was something where we knew we had to do better. And so we sat down and we said, okay, we need to build automation that can in real-time understand speech and understand whether it's toxic or not in what categories it's toxic and do it a fairly a high accuracy level, right? Precision recall had to be good. So that was back in December. It was a problem that we didn't think had been solved. The approaches that had been taken at that point were kind of, they were very slow. They tended to involve speech-to-text, text into a text filter and some result out. And that just wasn't going to work at the volumes we had and at the performance we wanted. Fast forward today, we have rolled out in English a real-time voice moderator that today will give you a slight push, a slight nudge. If you're not talking appropriately on the platform.

(07:48): They’ll say, wait a sec, maybe you need to think about the guidelines and what you're saying. Right? And we found this has an incredibly positive impact on users just doing that. Of course, the same system can give cues to places we need humans to moderate. It can generate automatic abuse reports. There's lots of applications for this. As precision accuracy go up, we can make this more and more automated. But even this first touch has dramatically improved the rollout we currently have in English with voice. And what's exciting about it is it is direct voice to action and classification. And what's been really useful about that is there's so much implied in your tone of voice about how you're behaving and when you do the old approach, you just lose all of that. So we feel very good about these results. We're currently looking at how we expand this to other languages, how we continue to improve the accuracy and it's a program we feel very proud of. It's been very successful so far.

How Roblox is sourcing its AI training data

Pete (08:42):Yeah, digging into that a little bit on the accuracy and around language models, and that always seems to be the big challenge in AI today, but just as you think about some of the things that you're looking for with this application, what are you using for data? Where's the underlying data coming from? Is that from Roblox itself or how are you training the models?

Daniel (09:03):
We use a mix of what we consider properly sourced open-source models where we know where the data's coming from and then almost always with refinement based on Roblox data.

Pete (09:12):
Got it.

Daniel (09:13):
In this case, of course, we had moderation examples, both from text and the ability to generate synthetic data using AI models to create synthetic voice data. And those models are kind of crazy. I've heard how we're kind of able to translate some offending statements into teenager. It's really kind of amazing to see the models do this. I would try to imitate it here, but I just wouldn't do it justice. But it's hilarious but also incredibly accurate. And then also of course the moderation decisions we're already making on the platform can go into these moderation models and we have so much of that. We have a really rich, unique set of data on the platform from all of this that enables us to build models that really work.

Pete (09:52):
That's cool. Actually, that translate teenager for parents might be an interesting other application.

Daniel (09:58):
Right? You could speak into your phone and your kids might actually understand what you're saying.

Roblox user demographics, and how brands are leveraging the platform

Pete (09:59):
Alright, got it. Very cool. I to back us up again, personal curiosity, but just thinking about the demographics, the age groups on, what is the sort of range in terms of the typical Roblox user or I guess the variety of that or the most engaged users on Roblox? What does that look like?

Daniel (10:18):
Right. So right now Roblox has really aged up, and has gotten a much broader demographic. Our fastest-growing demographic is 17 to 24. That doesn't mean we don't still have a lot of enthusiastic, younger players on the platform. We of course have a fast and quickly growing 25+ group on the platform, but we're really seeing incredible traction in 17 to 24. And we have the gamut from younger all the way up, which is why civility is so important to us and having age-specific civility. We just released, for example, just a few months ago, 17+ experiences on Roblox that you have to be ID verified 17 or older in order to go enjoy them. But a lot of our creators went right to that, whether it's a slightly more graphic video game or something else with more of an adult theme that's totally possible on the platform at this point and has loyal followings. So it's really broadened a lot and we saw that just over the past few years. I think Covid contributed that in part, people are looking for a new way to engage with each other. We've embraced that and look for ways to enable all of those communities to come together and enjoy 3-D interactive social experiences.

Pete (11:37):
The brands themselves, do they put out, I guess the digital versions of things that they have available, or coming available? Do they actually take creator inputs and design things around that? 

Daniel (11:51):
There's been a few of this. The brands primarily have started with this just trying to build their brand experience onto the platform in a 3-D place where you could interact with their products. Gucci Gardens launched a few years ago and that was just a huge hit where they were selling special limited Gucci items that their fans were excited about. But the cross between real world, physical world, and virtual is definitely there. The Parson School in New York did a whole design class where they translated real-world concepts - on Roblox - into digital designs. I was fortunate at our latest Roblox developers conference to hang out with some of the folks who ran that program, and it's just really incredible how the whole concept of digital design is becoming a form of its own. You really could start to see all the stuff starting to take off.

Daniel’s approach to finding talent, and sourcing contributors from the Roblox community

Pete (12:46):
One of the questions we definitely hear a lot when we're working with founders just on where is this talent going to be coming from? Or if I need talent or folks that are experts in certain areas that are new or emerging, where do I find them? I'm just curious how much or how you've gone about that as you've built out the teams just around a lot of the new generative AI work that you're doing. A lot of the platform stuff that's sort of changed with the release of this technology.

Daniel (13:13):
So I think it kind of backs up, and this conveys actually some of the things you and I wrote about in the blog post way back, but we start from first principles on talent. We are independent of discipline looking for folks who are incredibly creative, very strong problem solvers, self-actuated and driven, and I think that is probably the higher order bit on everything we recruit for. Then you can say, okay, well what particular technical skills do we need you to find? And look, there's a huge demand for AI talent, but there's also quite a few folks who've been attracted to that space and are available and are out there. And I think it's just really a matter of telling people why working with your platform is so incredibly interesting. And we've done so many cool things with AI.

(14:01): We have opportunities, like I mentioned, the voice data. Most folks aren't working with that. It's a new opportunity to do something new and groundbreaking in that sort of environment. And that's been our key. When we did the blog post, one of the things we spoke about was this whole idea of don't have too long a list of requirements. So you're just going to end up with mediocre across the board. We really believe that at Roblox, we focusing core on the capabilities we want. And yes, some core technical capabilities are a core part of that, but we try not to broaden that too much whether we're looking for a new college grad or whether we're looking for a senior executive. 

(14:52): Now, an interesting thing we had done, just to touch on that since I mentioned new college grads to operate at scale, we purchased a company a few years ago called Imbellus, to help us build a verifiable, accurate online assessment for all employees, but starting with new college grads. And we've had incredible success with that today. This assessment is done in Roblox. It's generally a problem-solving sort of activity. It allows us to open up our assessment to anyone at any university. So we are no longer limited by where we have enough recruiters to send to different schools. We have a Roblox career center online in Roblox that I have given talks to students that we've invited, but from all sorts of different schools, not all the ones we would be able to visit, not all the usual suspects as they might say, where they, therefore, can hear a talk on where Roblox is going from a technology point of view, ask me questions in real-time, hear my answers and so on. So I think the ability to apply Roblox and tooling build around Roblox as a way to better recruit.

(15:49): We’re absolutely living that with our new college grads and it's proving very successful. It's really allowed us to hone in on folks who are incredibly talented without getting fixated on the name of the school, the GPA, all that sort of stuff that are normally things that can lead to a more narrow pigeonhole sort of recruit.

Pete (16:09):
Well, that's often been the complaint, just you kind of go where you're used to going and you can end up with very non-diverse teams. Very, yeah. It may appear even from the outside to be somewhat diverse, but it's a bit more groupthink, right? It's not as diversified as you'd like.

Daniel (16:28):
Some of the strongest technical contributors to Roblox came to us directly out of the community, with no schooling, etc.

Pete (16:34):
Right. That's what I was going to ask. Yeah. So do you guys leverage that as a creator platform, right? 

Daniel (16:38):
We do. I mean, we are cautious about it. We don't want to steal great creators out of our community, but for some folks, this is the way they want to go and we want to be there for them. And when I think through the folks who have joined us that way, it's just such an incredibly high-performing group of people who just have rockets on their back from a career point of view, and they're doing incredible things every day.

Pete (17:01):
Well, just an unfair advantage, right too, I guess, of seeing the passion they've already got as they're on platform building things and then to be able to see if they're capable from a technical standpoint.

Daniel (17:09):
And likewise, we're a mission they're excited about and we know that and they know that, so they're excited to be part of it. So it works both ways.

Generative AI’s impact on Roblox, and the future of work

Pete (17:16):
Yeah.Some of the specific tools too. It seems like, and I was tinkering a little bit, can't say I'm good at it, but was just messing with some of the generative AI capabilities that you guys have built out. So as you look at some of those things, what are some of the more exciting ones that you're interested in or I guess kind of releasing to the world?

Daniel (17:36):
So last March we released two tools. We released a Material Creator and Coding Assist. Coding Assist is a little bit easier to explain, so I'll start there. Coding Assist is like copilot, but its purpose is for Lua coding in Roblox. And just like I mentioned with the voice, we've trained that both on open source models, but then also retraining on open source code we have on our platform, and I keep saying open source, we've been very careful with our own creators to make sure they opt-in on anything we use when we do this. But even then we have, first of all, our community is incredibly passionate about making everyone better. So we've had a lot of code volunteered, a lot of open source code that we've been able to use in that regard, but it really has sped up coding on the platform. We find folks using it about 2x as productive in how quickly they build code out as those who are not. Then I'll talk about Material Generator. This really, I think, enabled something that was out of the reach of most people before. So about a year before that we released a new sort of material graphics. Anyone in graphics or gaming will know this. They're called physically based rendering materials. PBR. Right. And the way they kind of work is rather than just having, let's say one-bit image that's kind of like a color map, you kind of have multiple layers that talk about how the materials should interact with light in the environment. So an experienced skilled creator can go build any of their own PBR materials, you can find them on the web for download and so on. But it was really, let's say it was totally out of my ability to go build one of those. I'm not an expert with these tools. We launched a generative AI tool where you can type something like hot bubbly lava and get a bunch of examples and refine, and I can now go build, anyone can go build a PBR material. That's exactly what we want. Whether it's a pink metallic for a car you're building, whether it's the gleam on a sword with a realistic metal color, or that lava example I gave. It's really very, very straightforward to do something that used to require extreme technical skill and therefore would probably not be done to now in the hands of almost any creator out there.

Pete (19:50):
Very cool. Continuing down that path a little bit too. So as you think about, I think a lot of what's being brought up today around AI, you think code-assist or suddenly now I can generate things that I might not otherwise have been able to do. What does that do to the folks say writing code or those creators or designers who were originally coming up with that content? How do you see the generative AI tools sort of impacting what people do or how they do it? Or I guess the big question, what does it do to people's jobs?

Daniel (20:24):
I actually think what it's going to do for creators is it is going to unleash them. I view what's going on with generative AI, not in any way a replacement for creative spark or creative ingenuity, but it just takes a lot of the drudgery and mundane tasks and moves them out of the way so you can create faster. And increasingly what it will mean is the success of a creator will be about that spark, that idea that they have. And then our job is probably the tool. So when they have that idea, they can move as fast and as reasonably possible in bringing it to life and being able to iterate on it and so on. So I'm incredibly bullish on this. I sometimes liken it to - imagine you're a farmer and you can have an abrasive oxen, or you can have a tractor, you're going to be a better farmer with the tractor, let's just admit it, right?

(21:10): That's what I view these creative tools as, and that's how we're going about designing them. It's not as a replacement for, it's an accelerant for, and that's our entire philosophy around this. As we bring in a Roblox studio, which in case folks listen, you don't know, that's our development toolkit for folks to download and use. Or eventually, probably we will enable third-party tools to interface directly with Roblox. We're opening up all our APIs so that becomes more possible. So you might build an incredible four Roblox car building AI someday and be able to plug it directly into Roblox. But I just view this all as an accelerant. It also opens a new sort of creator to the platform.

(21:47): I expect where it will go, is in a sense, if you are a really sharp AI builder, you can build a specialized tool that then other creators can use, and we'll give you ways to monetize that, and connect with creators as well. So part of our goal always is to bring every sort of creator onto the platform and there's a way to bring yet another sort of creator onto the platform. So I'm extremely excited and bullish on what generative AI means for creators on Roblox. I think it's just going to open an incredible age of even more fantastic creations.

Pete (22:20):
Yeah, I agree. I think we're in that phase. It just as a new tool or something becomes available, the sort of initial fear of what does that mean for me if it's sort of directly impacting what I do, but then ultimately where the value comes from, is just the creative side.

Daniel (22:34):
And look, we've seen that with every major step in technology. People initially worry and then we think, how did we live without this? I think we're going to find ourselves in the same place there.

The Roblox developer community, and how it’s evolved over the last four years

Pete (22:44):
When going back four years, as you think about what brought you to Roblox, what you liked about it, what's one of the biggest surprises I guess from today, or something that you just didn't see coming, or maybe that's everything.

Daniel (22:58):
I guess when I joined, I definitely believed there could be this broader vision out of games into all these other different ways to interact, but I don't think I was creative enough on all the different ways that could end up going and how incredibly fantastic our creator community is to work with. So now to admit, we just did RDC about a month ago, less than a month ago. So I'm coming off the high of hanging out with all those creators, but just a fantastic community and the ways they're taking the platform are just magical. And I think I had some inkling of that at a very almost superficial level when I joined. That's been the biggest surprise and incredible upside since I joined.

Pete (23:44):
What's been the growth that the conference of developer community in the last four years? I'd be curious. It seems like you'd have a mix of folks that are thinking about it from a tools perspective all the way through the creator side.

Daniel (23:55):
It's really exploded. I think the biggest difference that I've seen over the past four years is really the advent of dedicated studios. Companies. Into building on the platform that involves tens to maybe a hundred people working together to build Roblox content. Sometimes their own ideas, but sometimes also we mentioned the brands working with things like brands to build out content. So a lot of them were started by people who were just started as Roblox creators back in the day, and some are others who see the community and decide to step in and build a company around it. So I think that's been the most notable change I've seen. Our communities of creators is aging up along with the users, which makes logical sense. Some of the stories are incredible. There's one creator is they're a family that just build together as a business. Folks find it very accessible, very entrepreneurial. One of the great things about Roblox is it requires no capital to get started really, because we provide all the compute services, we provide all the tools, all the stuff. It requires your own time, and therefore there's relatively low friction if you're good at this to just kind of build and build and build.

Pete (25:19):
But I guess any creator, I think there's always what seems like this sort of instant, I built this thing and now I'm a YouTube celebrity or whatever.

Daniel (25:28):
Exactly.

Pete (25:28):
It's like most of those folks put in a significant amount of time and effort too just to go and build that brand and that platform.

Daniel (25:35):
That's another relatively new addition I never would've expected, which is the role of how much influencers have gotten involved with Roblox, right? That's a world I'm not terribly familiar with, but now they're a notable portion of the folks who attend RDC and a community that's just fantastic to engage with.

(25:51): Another interesting thing that where this is going, and we've just started on this path, but the idea that everyone can be a creator on a platform that we blur using and creating with what we're calling inexperience creation, where more and more you'll be able to just create from within experience. We've already seen this with some kind of artistic-oriented experiences where they're able to create things in the experience and save it back onto the platform and make it part of their avatar. We're doing some work right now with photo two, avatar creation where inexperience, you'd be able to use AI to generate a fully featured functional AI that looks the way you want it to based on a photo, possibly a text prompt as well. So imagine we wanted Dan looking like the joker, for example, and you'd be able to build that automatically and with it fully rigged and skinned avatars that you don't have to worry about all the technical complexity. And that means things like your eyes will blink, your mouth will smile, your eyebrows will move.

(26:51): All in response to your voice and your facial expression of your camera. And the creator doesn't have to know how to build any of this. It just kind of happens. What they know is they want Dan as the joker or whatever it is, and the style they want with the image they want. So we're very excited about that. That's something we're probably going to be launching in the next few months. But it starts to open this door where everyone is a creator, so we expect to go from avatars to arbitrary 3-D creation, another very hard open AI problem, particularly when you take into account 3-D creation. Roblox means it has to be kind of physically real. A car has to have moving tires and a steering wheel, not just an image of a car. So all that stuff is going to make, it makes it really interesting from a scientific engineering point of view, but also incredibly exciting about what it can mean for everyone on the platform.

How founders should think about scaling their teams in the age of AI

Pete (27:43):
The audience for Accel is a lot of founders starting early things. I would say that building teams is probably the most talked about thing, the top of mind things thing that keeps people up at night. But as you just think back to even your experience scaling Roblox and then even how we're sort of adapting to new technologies, what AI means to people today, but what advice would you have for a founder, say thinking about just strategically, how do I put the right things in place to scale a team? Any higher level advice and anything that might be changed now based on just where technology's going?

Daniel (28:26):
First of all, full disclosure, I've never been in their shoes. I have a lot of friends who are founders. I do advising of course, but it is tough to be a founder. It's like nothing else. And I really respect everything they do. I think the bit of advice I can give them, things that have worked for me as I've built up teams are first and foremost, and this is the hardest thing, don't be in a rush to close any rep. Yes, you have to make steady progress. It's really hard when you have a timeline, your company's got to build on, but that is where I think hiring mistakes happen. Hold out for the right individual. A key part that goes hand in hand with that though is really thinking hard about what is the right individual.

(29:06): And I've seen people make that mistake time and time again where they have some idealized version of what an incredible engineer might be, or great head of marketing or something like that. And I think you've got to get real with yourself about what are they going to do? What do you need 'em to do for you, and what sort of person do you want them to be? I don't think it's ever too early to be thinking about culture and what your company really stands for and be honest with yourself of how you operate as a founder. And I've seen folks think they can get to that once they're no longer going to die, once they know they're coming to success. And by that point you're enough, it's really, really hard to set the culture tone you want to set. And some of the founders I've seen who I think are building the best companies, have thought very deeply about what sort of people they're building it with, not just like what the product is.

(29:56): So if you have those two things, I think you're in really good shape by and large in that you know what you want your company to be. You've really honed the, ‘what are we looking for’ probably coming out of this. Okay, now how am I going to figure that out when I talk to someone for an hour or a few hours, usually over Zoom these days or whatever, it's a little bit, I always try to do in-person interviews now whenever I can. I feel I get so much more of a read off someone in person than I do over any electronic device.

(30:22): But those are what I'd recommend. And again, don't be in too much of a rush when I have an important hire. Probably the last VP of engineering I hired at Roblox most recently, I set out with that search being a year-plus search. And it took about that long and I did not rush myself, and I met a lot of folks who were just not the right fit for me or the team or the company,even though they were incredibly accomplished individuals. And then we were able to find the right person and that was great. So don't be in too much of a rush. 

Pete (30:57):
I think patience is key. And I know obviously as investors, I think there's often a push to move faster, but I think for sure when it comes to hiring, doing the right thing, is often more important than doing the quick thing. 

The war for AI talent, and optimizing teams for emerging technologies

Daniel (31:10):
It also ties to look with the war for AI talent, it's hard. It is absolutely hard. It's even hard for Roblox to find the right people, sometimes, it's even harder when you're a startup, no one's heard of, but this goes back to being patient. When you find someone who is incredibly lit up by what you are trying to build, you’re going to find it's a lot easier to recruit them. And I think that's an important thing. Start in your recruiting process by making sure they're excited about what you're doing, or maybe they're just never going to be excited about what you're doing. And then it doesn't matter how good they are, they're probably not going to your company. And if they are excited about what you're doing, I think you can find it's a lot easier to bring them in. And it might take a little bit longer to find someone with that criteria that they're excited about what you're doing. But you know founders, they're nothing if not enthusiastic about the product they are building, the system they're building. And so that's usually infectious. So you will find that person. And once you do that, then it's just a matter of testing the other parts and making sure they're a good fit for the skills you need to have, the cultural capabilities you need to have. But once you've got 'em hooked on your company, you've got that time.

Pete (32:20):
And is it just the fit and the raw intelligence that you would use? I guess where I'm going with the question is more how do you optimize around, say, a new technology where there's not a large pool of expertise? So what do you look for in the sense of, okay, if there's not five years, 10 years of experience with say, generative AI, then what are the core skill sets that you look for? What would you recommend?

Daniel (32:42):
I think you can never go wrong with smart and humble people. I think they can get a lot done, and there'll be many times in a career. I know in my career, you have to be a lifelong learner. And I think hiring people like that, particularly into a small company, which you don't know what your company's going to look like a year or two years from now, makes a ton of sense. And so one of the things I always look for, the Roblox always looks for is just overall, does this person have the drive and the creativity and in some degree the intellectual horsepower to do whatever we need them to do.

(33:19): There are times where it's great, yes, you want to layer on the particular skills, but if that's where you start, then I think you're going to find yourself very limited, as soon as you should take a left turn. I was talking to a manager the other day who was talking about a challenge they had where they hired someone who just wanted to program in one programming language and then they needed to change their project. And this all of a sudden was an issue. I was kind of like, oh, that probably wasn't the best hiring move to make in the first place. And I find the best people, particularly in a small environment where you got to move fast, is people who just generally have a lot of curiosity and drive.

Pete (33:53):
Yeah. And I think we've seen that happen where oftentimes you compromise for the expertise on something else, and you end up with often a poor cultural fit. It's like, well, I think I'll take that person because they're very good at this, even if ultimately it might not be a great fit, and it just doesn't work out.

Daniel (34:10):
Now, to be fair with AI, depending on what you're doing, if you're really in a company where you need to drive the limits of artificial intelligence, you need people who have probably the breadth of theoretical knowledge behind that. But I think that's different from has this person exactly worked on this problem before? And I think it's challenging enough, you just say, look, I want someone who's just very strong in the fundamentals of AI. You've narrowed your search quite a bit already. And I would look them for other characteristics on top of that rather than, oh, they built exactly the model I want to build somewhere else.

Pete (34:43):
How quickly can they learn? How quickly can they adapt to things?

Daniel (34:45):
That's right.

Daniel’s POV on future AI challenges

Pete (34:46):
Got it. Alright. Yeah. More on the, I guess, topic of AI. As you think about just what are the things that would keep you up at night around that? Just the concerns, the questions. I think you can get into a lot of different topics around this subject just as it comes to all kinds of things, really in terms of what we do with it, how it's applied, when will the robots come back from the future?

Daniel (35:10):
Yeah, that's right. That's right. I think at least at Roblox, we're fortunate that we've always had a strong tradition of content moderation, content safety. Also, because we're a three D immersive environment, a lot of the things people are rightfully worried about in society, like a deep fake are kind of like a feature, not a bug in Roblox, the whole point is you're an avatar and you're not being yourself. And that avatar may or may not be super realistic, but however you want to design it. So that's kind of all, okay. An area we do think a lot about is how are models built and what is the origin of the data that is being used? We really believe strongly in respecting the IP of our creators. And so we look very long and hard at any model we're using. Where was the model's data sourced from? What does that look like? When we use data from our platform, particularly for anything that's going to be creative, we are very careful to make sure it's always opt-in in that regard.

(36:09): Things like moderation models are a little different. They're training to detect bad content. Generally, people aren't going to opt in their data as bad examples of something. But again, we're not using that to create or generate work when we do that. So anything that's generative, we absolutely focus on the lineage of that data, and we consider that take that very, very seriously. I think it's an area where sometimes it's very difficult to even know. A great open source model appears on hugging face and maybe, you know, don't know. It can sometimes be clear, sometimes it's not. So you have to be very careful with that, and it can mean you have to take a step back sometimes and think before you move forward.

Barbeque, keeping it simple and closing thoughts

Pete (36:51):
Well, the last question is completely unrelated to Roblox and technology is, I know Dan's a bit of a barbecue aficionado, so I'm always interested in sort of what's the latest thing. So hardware and software, I guess, what are you using right now? What are you most excited about in the world of barbecue?

Daniel (37:07):
Yeah, so do we have another hour?

Pete (37:10):
Yes.

Daniel (37:10):
Right. I've kept my hardware really simple, and I'd say the only thing I've really added in the past few years is now that my wife and I are empty nesters, I've actually also brought in just a simple Weber kettle, and I do a lot of smoking on that one. It's just for the two of us. I don't need to fire up a big smoker when I'm just cooking for two. Otherwise, from a software recipe point of view, what I've decided on the meat itself, I've gotten simpler. I do much more just salt and pepper, maybe salt and pepper and paprika, depending on what I'm doing. Keep it really, really simple. And then where I've gotten more exploratories on sauces, I just started making my own sauces and I found, wow, you can put things in there that just really no store-bought sauce is going to have that mix of flavors you want, whether it's, I like a little bit of bourbon in a lot of my sauces, but not too much. And I found an ingredient that I think makes the sauce great. We have a really good fish sauce in the house, like Southeast Asia, and that really adds an umami hint to the sauce, which I think is fantastic. So the ability to control all that, I think is a lot of fun. And keep the meat simple.

Pete (38:21):
Yeah. I love it. Well, Dan, thanks again. It's been great having you here.

Daniel (38:26):
My pleasure.

Pete (38:26):
Just sort of reminiscing on some of the old stuff. So excited to come full circle here and just get the update and hear how you're thinking about the hottest topic today, AI. So thank you, Dan. Thanks for having me again.