Simulating Battles to Test Abilities / Opponents
This is one of the gauss weapons in JAGS Have-Not, artist rendition.
This is the 3d render I did to show what the gun "really" looks like!
(the images are mostly unrelated to the point of this post). One of the things we are doing is using a java-based simulator that reads data files from an excel spreadsheet and uses them to create groups of opponents. Usually Level X PCs vs. one-or-more "monster." We then run 5000 combats and see:
What % each side won.
How many battles out of the 5k did each side take causalities (and how many)
How long did the average battle take in Rounds.
In real life a group of PCs can vary a lot. For one thing, there might not be 4 of them. For another, some character concepts don't fight well at all--much less in a semi-optimized situation. Finally, even if there's a group of PCs who are pretty combat-worthy, they usually have different styles or weapons--and the simulator doesn't take that into account (to be clear, it can--and we have done that for testing Archetype Powers which we used in a "supers game" style environment)--but for your fantasy or post-apocalypse monster, we didn't see much value in varying the sample PCs since we didn't know what we'd learn one way or the other.
We also had to determine how often we wanted Total Party Kills (TPK). A TPK in a tabletop game is an end-condition that usually doesn't satisfy players. We've had a few now and again--but they are not the equivalent of a TPK in a computer game where you just restart. They represent a loss of valuable collateral (time spent with the character, the GM's investment in campaign materials, the value of the imaginary group history, etc.).
Some groups dig that--many don't--but whichever the case, we had to literally code it into our stats generator: how often should a boss-fight end in the death of the party (note: there are many boss fights where the party being unconscious wouldn't mean the character's deaths--we're aware--but the question is still relevant).
We settled on 4%.
A good "boss fight" should have a 4% chance of wiping out the party--with the chances going up to 6%-8% before we decided "the boss" was higher level." (NOTE: if you think 4% is wrong you can just run lower or higher level bosses against your PCs and adjust the TPK rate--although it may not be entirely predictable as to how that will work--often doubling the bosses resulted in going from a 3% or so TPK chance to like a 78% TPK chance--especially if the original fight was lengthy and two of those opponents dragged it out past what the PCs could handle).
So, like I said, a lot of questions (how long should an average fight last? We figured 2-5 Rounds was about right--if something was taking 7-12 we considered it a pretty darn special case and probably needed adjusting).
I'm sure that some other game designers have used computer simulations to create monsters (I would expect the 5e guys had the resources and wherewithal to do that) but I'm not aware of any of it or the specifics of how they made those decisions. This was a new frontier for us.