Our community blogs

1. 

   Torque 3D

   A group blog by DEVGRU in Work Blogs

   • 9
     entries
   • 19
     comments
   • 79093
     views

   Recent Entries

   • Mid-2026 Workblog, part 1!
   • Rounding the Bend
   • First update of 2025!
   • End of March-ish Workblog 2023
   • Workblog Nov 2022

   

   Latest Entry
   

   Mid-2026 Workblog, part 1!

   By JeffR, Saturday at 05:40 AM

   Greetings and salutations!
   It’s been a minute since I’ve last yapped. Certainly a thing that happens more than I’d like, but it’s also easy to let oneself get distracted by chasing down bugs, trying to do refinements, or features, and just generally getting improvements and updates to the engine done.

   Which, of course, is why people like reading me blather on. Find out about all the neat stuff that’s been worked on, rolled in, or is slated for future developments.

   So hey, let’s go ahead and dig into some of that stuff!

   
   So, what’s new?
   Oh man, quite a lot! Let’s start by doing a quick breakdown of some of the PRs that’ve been rolled in!

   All told, we had around 200 pull requests merged in, covering pretty much the full spectrum of things, from rendering to networking, AI to library updates.
   Since there’s so danged much, I figure I’ll just snag a few of the interesting ones to go over in specific:

   
   Rendering

   • Rewrote GBitmap to better support for formats, cubemap images, and texture arrays
   • Updates to probe data and some BRDF/IBL tweaks and improvements
   • Fixed up ribbon particle seams bug
   • Reworks to the wetness shaderstuffs
   • Fixed up some shapes’ surfaces being erroneously shadows wrong
   • Compiled shaders will now generate out a TSO file to help reduce shader load times where possible

   Physics

   • Updated Bullet library
   • Fixed a bug with ConvexFeature::collide that could cause collision to just be dropped
   • Fixes for player class having a bug with a wrong var in the updatePos collision loop, as well as better handling for degenerate polysoup collisions so players can’t as easily clip through them
   • TSStatic using the fallback shape now doesn’t break stuff with collisions

   Assets

   • Image, Shape, and Material/TerrainMaterial assets converted over to utilizing AssetRef structure, and removal of the old DECLARE_ASSET-style macros for those, along with standardization of formatting/usage of setters/getters for actual using-classes of said asset types.
   • Updates to Assimp and multiple fixes to deal with axis correction and other problem behaviors with FBX/GLTF due to DAE support being removed from all modern blender versions
   • Cached.dts files can be encrypted now, and with those changes the version number of DTS has been increased from 28 to 29.
     • Important to Note: The version change can potentially cause older builds to not play nice if you try and bring in a newer version of a dts file into an older engine build without doing the updates. Also, if you have the project password enabled, then obviously the files won't load into another Torque build unless that provides the password the shape was encrypted with.
   • Better handling of parsing DSQ-vs-DTS datastuffs
   • Fixed a bug with cleanup/teardown that would throw an exception on exit due to the hasAccumulation checks
   • Networking of asset stuffs has moved to hashed Id’s instead of full strings, heavily reducing network transmission
   • ImageAssets can represent and be used to ref named targets for consistency

   Editor Stuff

   • Rebuilt the Particle Editor to now utilize the standard guiInspector controls instead of a fully custom adhoc gui, similar to what was done with the Material Editor previously. This means it’s much easier to change and expand editing of particles, emitters, explosions, etc, by just changing or adding fields to those classes’ initPersistFields, same as everything else, now.
   • Retooled editor camera mode so it’s easier to expand to have new modes. For example, the AAK module will now list the AAKCamera as a valid camera mode in the popup list, and respect being toggled between.
   • Various tweaks and fixes to the ShapeEditor gui

   Scripting

   • Default arguments values are now a thing in TorqueScript
   • FieldDescriptor builder api to more clearly and cleanly define out class fields using appending functions instead of a monolithic multi-param function.
   • Better error tracking and reporting with exec calls, so it’s easier to pin down when something goes wrong if you’re executing logic in the console, or a command field, etc.
   • Optional arg to eval() to have it just dump out what string is being evaled. Good for debugging

   Audio/SFX

   • Major update to sfx to facilitate new features in the future like HRTF support, reverb, etc
   • Crash fixes with blank sounds
   • Network update bug for SFXEmitter
   • Fixed channel 0 being muted by default

   AI/Navigation

   • Fixed a cover-radius calc bug
   • Added additional state checks for AI movement for if in the air, water, etc.
   • Various improvements to the navmesh tooling and creation for stability and consistency

   Game Classes

   • Reworks/expansions to Player class to rename hitboxes, add prone mechanics, turn animations, etc
   • Added ability for derived classes to expand the actionAnim table
   • Fixed some script execution behavior for the DamageModel module for better consistency

   UI

   • Various fixes for the BaseUI module. Keybind persistence, mouse sensitivity slider, key remapping fixes, etc
   • Update to guiPopupCtrl to fix handling of | characters
   • Fix for positional drift with multiple GUI resizes
   • Fixed a divide-by-zero error with scrollbar
   • Controls menu/refresh bug fixes

   Core Engine Stuff/Libraries

   • Refactored RefBase to work with Smart Pointers for better, more stable memory management across the board
   • Updates to SIMD matrix math stuff
   • Restored TAML reading/compilation for binary and json formats
   • An XCode-specific bug that dealt with global statics being destructed incorrectly
   • CI pipeline cleanups
   • Libpng re-integrated
   • Added vcpkg logic to the cmake build process. In large part because libSndFile library requires it to be built correctly and not cause a licensing crisis. Also adjusted to beter target strict library package versions and, if possible, only use the local/platform versions to avoid security vulnerability threats and keep things stable and consistent
   • Shadergen can produce a ShaderData class, minimizing shader rebuilds when macros change
   • Filestream can now write asynchronously to not stall the main thread
   • Work on precompile headers usage to boost compilation speed

   Neat! So, how ‘bout that 4.1 release?
   Yep, yep! It’s been a slower cook than we’d wanted, to say the least.
   One part ‘R&D doesn’t have a guaranteed baking time’ and another part ‘getting distracted by fiddling and working on stuff that DOES deserve eyeballing, but maybe not so much a priority problem’.
   As it stands, the biggest question is, of course, “What’s left to do?”
   Fortunately? Not much. 
   To wit:

   Finishing out Asset Type Conversion
   I’d mentioned in the what’s new section about Shapes, Images, and Materials assets getting converted over to utilizing AssetRef stuff. To go into more detail about that and what that means:

   For the end user, like mappers, artists, etc? Basically nothing changes at all. The exposed field behavior and script functions, inspector fields, etc are all mechanically the same. It should by and large be a wholly seamless transition you shouldn’t notice.

   For people working in the actual code, it’s generally an improvement and cleanup.
   So, if you’ve fiddled with the classes that use assets, you may have seen macros like this before:

   DECLARE_SHAPEASSET_NET_REFACTOR(TSStatic, Shape, AdvancedStaticOptionsMask)

   Which isn’t exactly a super clean, ‘rolls of the tongue’ bit of code unto itself, but it’s actually a macro that does a LOT of stuff under the hood. Generally useful stuff, in that it sets up a bunch of boilerplate, utility methods, etc, but macros aren’t exactly fun to debug at the best of times, not all classes need all the bloat of the boilerplate, and everything was kinda built around protected methods for exposed fields which requires custom setter/getters every time, which is MORE boilerplate, etc, etc, etc.

   And when I say that they do a lot and expand into a lot of code boilerplate under the hood, I'm not kidding. This is what that one line above turns into for actual effective code:
   

   Whole WALL worth of stuff, which is also in the class header file, and so it even makes compile times longer!

   And then if you want Network-aware fields, that’s it’s own macro, like above, and then if you need an array of asset, like Material having an array of Image Assets to cover each stage, that’s ANOTHER macro, etc.

   So even with the ‘convenience’ of the macros, you were also looking at a lot of code duplication. Duplicated code that’s just not that easy to debug if(or when) something goes wrong, so on and so forth. So instead we shifted things up to a much, much leaner structure that was more targeted, easier to understand/read, easier to debug.
   More memory efficient.
   More robust to failure.
   Easier to use more directly in the actual classes.
   Easier to have work with fallbacks, not breaking if you accidentally typo’d something, or an asset fails to load so it doesn’t just blank out the field’s value.
   Easier to do error reporting if/when something goes wrong.
   …
   You get the idea.

   So what does that look like? Well, instead of a half-dozen special DECLARE_ macros and then another half-dozen macros spread throughout a class to implement things, using TSStatic from above, it now looks like this:

   AssetRef<ShapeAsset> mShapeAssetRef;

   Tadaaaa.

   I mean, yeah that’s not *everything*, but everything now just sorta ‘clicks’ to the AssetRef type. It self-initializes to empty in a pointer-safe way, and there’s integration for the console fields so you don’t have to do anything special there, either.
   It just handles it all internally. Setting, getting.

   Then with the added field constructor stuff mentioned above as well, you end up with:

   ADD_FIELD("shapeAsset", TypeShapeAssetRef, Offset(mShapeAssetRef, TSStatic))
      .doc("Model to use for this TSStatic")
      .network(AdvancedStaticOptionsMask);

   And as you can see there, you just have the Type*AssetRef field type, and it will Just Work.
   Setting the AssetRef var will do validation internally, and if successful, contains a valid asset ptr:

   shape = mShapeAssetRef.assetPtr->getShapeResource();

   And you can do validity checks on it to make sure it’s all kosher first:

   if (mShapeAssetRef.isNull())

   If you try and assign it an invalid assetId because something went wrong somewhere, it’ll internally store what you tried to assign it in an internal StringTableEntry, too. So you don’t lose any data. It’s ultimately pretty danged hard to break it in the event something goes weird somewhere, and usage of the data is pretty straightforward now.

   As a little aside, the bit on the ADD_FIELD demonstration up there? With the .network() bit?
   Yeah, that automatically flags the provided Bitmask as dirty when the field is changed. So you don’t have to do a bunch of manual function defines to just prod the net update anymore either.

   All in all, significantly more straightforward, clean, and legible, right?

   Which brings us to the TODO.
   The last ‘big one’ to convert over is SoundAsset. The other remaining types’ll need it too, but those aren’t as extensively utilized in a bunch of places throughout the engine, so those’ll be super easy to add their AssetRef bits.
   Sound’s the last of the ‘major’ types, and it’s already underway. I just gotta finish out the bits of conversion and standardization, followed by testing to make sure nothing’s OBVIOUSLY hideously broken.

   Since at its core it’s just standardization work, there’s nothing truly sexy or showy about it, but stuff just generally falls into a “It just works” category to keep things stable and reliable in a way that actual developers don’t even need to really think about 99% of the time.
   And everyone likes not having to put active brainpower at unimportant things.

   
   Probes and Probes Accessories
   This one’s a bit more a ‘big deal’ for shipping purposes. The problem we ran into at-scale with the current probe system is that internally, the manager registers and… well, manages, all the probes in a scene. Which is fine, of course. That’s expected. Manager manages.

   The problem is that there’s not really any true shuffling of probes to keep a reasonable cap on the internal arrays/memory. 
   See, when a probe loads in, it self-registers to the manager, gets it’s cached struct of probe info, and loads it’s cubemaps for IBL into the manager’s texture arrays.
   If there’s no free slots in the arrays, the manager allocates more. This happens basically seamlessly and you don’t really notice any of these movies parts in normal circumstances.
   All very clean and automagic.

   Until you have a complex or large enough scene with enough probes.

   Then it turns out that graphics APIs have a limit on texture arrays!
   And violating that leads to all kinds of very bad, no-good happenings!

   So in a small level, it’d never come up, but in a larger level, or a level that uses a lot of SubScene loading to bring in more content, you could run headfirst into a hardware limit and nuke your graphics driver, which is definitely bad.

   So, it needs a refactor.
   Good news is, we’ve got a plan for it, and a solid chunk of work done towards it, namely: Atlases.

   

   “That doesn’t exp-”

   Yes, yes, shuuuuush. Lemme explain.
   So, the reality is, that unless you’re going reaaaaaaally crazy with probes, pragmatically a normal scene, even in a AAA scene, is only going to have a small number of active, full, IBL-providing probes in the immediate space around the camera. Largely because probes are supposed to cover a larger space, like a room, or a hallway, etc and not one ever 2 inches. 

   That’s a whole different lighting system approach.

   So, if we know that we only really need a small handful of “real” probes, then we only load a small handful of probes into the texture arrays to render!

   What’s that?

   “What about stuff far away?”

   Alright, you got me on that one. The problem is that that means you only get IBL/reflections on stuff close with the full probes. Which brings us back to the original problem.
   That is, unless we bake down all our probes into a super efficiently compacted data structure that can be more easily referenced at-scale.

   This is where atlases come in. You may or may not be familiar with the general idea, but if you’ve ever looked at a Minecraft block texture sheet? That’s an atlas. A sliced up grid of a texture, where each cell holds a reliable chunk of info.

   In minecraft, the cell at a given coordinate is always associated to a given block type, like cobblestone.

   In the new probes setup, each cell will represent a piece of data that describes a probe. A whole row is the full data of a given probe, including the compressed diffuse lighting info of said probe.

   This means that instead of needing to submit dozens of cubemaps into the manager’s texture array, we can just hand up atlases and then do lookups of a whole bunch of probes in a much, much, much more efficient way. Fewer texture binds, fewer uniform binds. It’s all in the atlas sheet and we just look up by index.

   This means we can render basic info for WAY more probes at a time, including much further away, which means we can keep the inside of a house looking darker, as one would expect, without having to chokeslam your GPU with a million cubemaps to allow it to happen.

   Efficiency up. Memory use down. Wins all around.

   And as said, good chunk of the work’s at least prototyped, so it’s mostly just about turning it into the ‘full production-ready state’.

   If you’ve already got a level with probes, all you’ll need to do is re-bake, and otherwise in theory you don’t need to life a finger otherwise, and just sit back and enjoy the improvements.

   
   What Else?
   Nothing.

   
   Lies.
   Alright, alright, you got me. Sorta. Kinda. Ish. These aren’t “blockers” in the same way the above two things are, and can be worked on in parallel as the RC gets its beatups passes without stalling the show, but I’ll go over them in part two, as this is already getting pretty long in the tooth. The good news is, part two is 99% written, I just wanna space it out so people don’t get flashbanged by an entire novella all at once in an update.

   So, see you very, very soon for the remainder for 4.1 things, and what’s the plan for the future!
    

   -JeffR

   • Read more...
   • 1 comment
2. 

   Torque 2D

   A group blog by DEVGRU in Work Blogs

   • 1
     entry
   • 0
     comments
   • 77044
     views

   Recent Entries

   • Torque2D: Rocket Edition Road Map

   

   Latest Entry
   

   Torque2D: Rocket Edition Road Map

   By greenfire27, August 10, 2021

   Early Access 1

   Today I've released what was held in the Development Branch into the Master as Torque2D 4.0, the Rocket Edition, Early Access 1. That long rambling name deserves some explanation. First, the we needed to get this code out. The big change is the Gui System update which I actually started in 2018. Most of the Gui System has been updated by now and it's been battle tested by the construction of the Asset Manager.

   Which brings me to the second reason for the release, namely, the Asset Manager. This newly minted manager is immensely useful for seeing all the assets and building particle effects. Unfortunately, the Asset Manager is still rough around the edges so that's why we're doing an Early Access version.

   If you're starting a game, or just want to play around in 2D, you should really be using 4.0 since it has breaking changes from 3.5. Plus, I would greatly appreciate feedback on the Asset Manager. The "Rocket Edition" is just a nice text name for 4.x which I'll run with until we reach 5.x. Mostly it's a marketing theme.

   So what's next for the engine? Let's review the plan.

   Rocket Edition Road Map

   Disclaimer: This is a map, not the Bible. I will likely make changes as we go. Also there are no dates by design. These things happen as we have time.

   • Early Access 2 - Remaining Gui System changes and Improvements to the Asset Manger such as a visual animation editor and support for named image frames.
   • Early Access 3 - Project Manager to allow the copying of modules from the Module Library and a couple of modules for said library.
   • Early Access 4 - New game demo based on the Rocket Edition, Perlin Noise Function, and Complex Colors on Tile Maps.
   • Version 4.0 - Update to the Toybox to better match the Rocket Edition.
   • Version 4.1 - The Gui Editor! (We have started this as a toy. Go thank Marauder2k9!)
   • Version 4.2 - The Scene Editor! (This has also been started as a toy, again by Marauder.)
   • Version 4.3 - Empty for now...

   That's the currently plan. I for one, am excited to see these things completed, but there's a lot of work to be done. If you're interested in helping, please reach out to me through Discord.

   Carry on, citizens! 

   • Read more...
   • 0 comments