Clanfolk update for 10 August 2022

Hi Everyone,

So I spent the last week in a nerd trance optimizing Clanfolk. The homesteads have gotten bigger than I ever expected and performance was suffering. It was time to roll up my sleeves and break some code :)

This update took longer than expected (8 days,) and I included all the changelists below so that you can see the fine details/suffering if you are interested.

The long story short is that Clanfolk now runs a LOT faster.

Please be advised, I reworked a lot of core logic and may have broken a few things. The testing branch team has been awesome in finding bugs as I made optimizations, but there may be a few left. I will fix these as fast as I can in the form of hotfixes.

I also plan to be here on Steam and on Discord more again starting today again. The optimization work was complex and intense, so I had to block everything out as I worked.

I also plan to take a day or two off soon. For at least the last 2 months I have been working 16 hour days 7 days a week, and I think that I better stop that for my long term health. I have been in emergency mode for release and then the follow up flurry of bug fixes, but now I would like to get into a mindset of a marathon again where I provide content and bug fixes at a regular, but sane, pace. In the long run this will also make the game better.

Starving Animals Fixed

I am calling this one out specially as it was one of the core reasons for the optimizations. I am sorry that it took so long. Some of the CPU gains that I made were applied to making the Livestock brains update faster. This should be the end of those trough starvations.

Large Homesteads Note

So Clanfolk can now comfortably support 40 humans and 40 Livestock. Eventually there will be some soft gameplay push back as populations get too high since Clanfolk is not designed as a city simulator. The AI is just far to deep to support a huge number of agents at once. To me 40 Clanfolk already looks like an ant colony, but it works now at least :)

Large Task Count (Fewer Tasks Are Better)

There are warnings coming for huge task counts. In some cases I was seeing 200+ Tasks per Unit which is just far too high. It is really not in your best interest to have that many tasks as all those tasks are constantly being evaluated by the AI as well as creating a huge work backlog for the units before they can get to lower priority tasks. They essentially get locked down.

People work around these lockdowns using Boost, but that makes things worse since Boost is an emergency tactic pulling all attention towards one task. Players who keep their task count low (300 or less) and use Boost sparingly report having a much better time and report the AI doing quite well in completing tasks.

V0.247

• Made Non Strict Priority checking more efficient
• Optimized task checking substantially such that checking for new tasks while other tasks are in progress happens less often, saving CPU
• Optimized all "feed object" tasks such that they will only see valid target objects before any expensive checks begin

V0.248

• Fixed problem introduced due to optimizations in v0.247 that prevented cooking

V0.249

• Upgraded haulable system to better handle large numbers of haulables late game
• Upgraded to a newer TextMesh Pro, saving CPU by disabling Text Scaling of all things.
• Optimized the roof support updater for blueprints. Saves a lot of CPU when there are lots of roofs active.
• Optimized window lighting
• Optimized entity distance searches around walls

V0.250

• Fixed oddly drawn windows and vents as a result of v0.249

V0.251

• Cached tool finding code for large numbers of construction jobs, large CPU savings
• Added new internal build task sorting system to prioritize and cull build tasks better. fixes many large hitches when there are lots of build tasks.
• Added cached string system to lower memory usage in the Jobs Menu (reduces hitching)
• Haulable system checks for haulable destination stockpiles over time instead of on demand (reduces hitching with lots of haulables)
• Upgraded the Tasks Menu Updater to make it take 5x less CPU.
• Improved memory management with tilemaps.
• Fixed broken baby carry due to an optimization in v0.247. No more floating babies
• Clanfolk will only go after spoiling food to eat with higher desirability if it is in a stockpile (no more berry parades)
• Applied some CPU gains to making the task system update faster at higher time scales. This should lead to more consistent behavior at high speed
• Added new high priority Livestock task manager to make sure that they get enough updates when eating when there are a lot of livestock
• Added an update multiplier to task manager based on current unit count vs reference count of (20) So at 40 units, there will be double the updates (and cpu cost) But things should run more consistently.
• Starving Animals At Troughs Fixed!

V0.252

• Updated French Language
• Fixed case where packing up an object on top of floor or roofs being built could disable those build jobs permanently
• Fixed cleanup of build tasks in the new optimized system that could lead to packup orders being ignored.
• Optimized PackupManager such that is tracks only the pack up object instead of all objects because it gets out of hand fast with huge homesteads.

V0.253

• Removed a bunch of spam that tends to bloat the log files without providing useful info
• Added a sanity checker to try to catch error cases with haulable lists and repair them
• Added possible fix that could cause a null entry to be added to the haulable list after an items destruction if timed just right
• Optimized the Attribute Updater to use less memory and skip inactive attributes.
• Fixed haul not calculating its relative priority to other tasks properly in non Strict Skill Priorities (everything was at priority 5)
• Upgraded Closest Entity finder to categorize items in such a way that very large item counts are less of a problem CPU wise.
• Upgraded Attribute Manager with mass item finding system
• All foods given an equal +2 priority offset so hauling of different food types is consistent but prioritized over normal items
• Update Clear Tile and Harvest Tile to segregate their tasks by related work skill to lessen the impact of large masses of tasks
• Time Sliced the task update manager to remove the 1 second hitch that was sometimes visible under heavy cpu load.

V0.254

• Fixed broken auto farming from V0.253

V0.255

• Fixed build tasks that require a tile to be cleared beneath them not showing up in the task list.
• Fixed task priorities on tasks on top of tiles that need to be cleared being ignored (was using the skill priority of the clearing task)
• Added better handling for paths that take a long time to return. Causes a lot less re-pathing also better task persistence
• Added new Trapped Detection that will trigger if a unit is in a room < 50 tiles with no way out. This prevents recurring expensive task finding calcs
• Added a Trapped Alert.

V0.256

• Changed: Babies trapped state is not checked vs doors since they tend to be carried through doors and don't carry keys
• Fixed non built crafting objects being valid for the auto supply checks, sometimes causing the auto supply to fail until the object was built.
• Fixed Seniors not getting private room bonus
• Fixed Bunny Dump at Butcher Block not working when the Butcher Block is reserved and there is no corpse stockpile.

Next Top Priorities

• Getting Hunters to Haul Bunnies when they are cold [DONE Update 2]
• Preventing personal needs from breaking chained tasks [DONE Update 2]
• Keeping fires lit overnight [DONE Update 3]
• Automating burying loved ones [DONE Update 4]
• Fixing bed owners stealing unclaimed beds, blocking them [DONE Update 4]
• Wells duplicating water jugs somehow [DONE Update 4]
• Upgrading the Boost System to also read skill ordering for decision making [DONE Update 5]
• Any and all cases of a movement stuck problem causing starvation [DONE Update 5]
• Performance upgrades for very large homesteads 40+ Clanfolk 100+ Livestock [DONE Update 6]
• System to make large task counts and over boosting less attractive to new players
• Carrying Tools back home better
• Cats and Rats

Thanks again people on the testing branch. It helps keep the main branch stable. If you like quicker daily updates, come join us in the testing branch too :)

Library->Clanfolk->Properties->Betas->testing

Diving back into the code now.

Andrew Hume (Blorf)
MinMax Games Ltd.