The real payoff to me is when someone asks why they didn't just try more ways of making companions unkillable without/before going through all that debugging. The response:
> that was our fallback, but
> -without actually knowing the cause, we couldn’t be sure that’d fix it
> -whatever the cause of the problem is might be causing OTHER problems, so we should find it
> -games near ship are fragile and it’s best to not make changes you don’t 100% understand
_THIS_ is someone that is practicing good software practices. I have no doubt this is why Outer Worlds is so much less buggy than, say, recent Bethesda _patches_. I saw a report this morning that the latest Fallout 76 patch causes some armors to degrade when a weapon reloads. This sounds exactly like the kind of bug you'd get from "fixing" something without understanding the actual cause, and thus just kicking up more emergent bugs because of your fix.
Complex interactions (like modern games), filled with shortcuts and assumptions (like modern games) means some bugs are inevitable. Requiring rigorous understanding before making changes can keep those bugs from multiplying.
...says someone with zero experience in coding games, but plenty of experience both fixing and generating bugs.
You must have a much better mechanic than usual. Most of the ones I've worked with tend to make a quick guess at what's broken, and just replace it. I call it "throwing parts at the problem" when it goes badly.
Though in defense, most of them are good enough to guess right 90% of the time. And mechanics generally get paid by the task, not hour. Replacing the probably misbehaving X pays. Spending an extra half an hour making sure that X really is broken and replacement is the best solution doesn't pay. So they just replace the X and hope for the best, and then replace something else if it didn't work.
Replacing a standardized part in a mechanism that is built entirely from such parts is quite different, though. You don't have to worry about breaking something - and that is entirely by design. This kind of thing is what makes the logistics behind our civilization scale so well.
That we do have to worry about breaking something in the software industry field, just tells us how far we've yet to go.
Agreed; it's a blessing. I'm actually keeping my brand of car as long as my mechanic retires, and I have 2 brands that I won't ever touch again due to bad experiences with contract garages.
I disagree. Placing a time cap on finding , fixing, and documenting the root issue of a bug incentivies beating the clock over working the actual issue. However, this all depends on the buisness goals for a software development team. If long term stability is not that important, then I can understand why a time cap for fixing a bug is encouraged. Ideally though I would encourage developers to find organisations that allow developers to understand the root of an issue before fixing it.
You can always do the both. If there was an "easy" fix that can be applied in a day or two, it should have been done.
Take all the time you think you can afford if yyou software is not released. But it is being used by customers and such a large bug is in it and if you are not reacting to it then imo it is a problem.
I think people disagreeing with me is unable to see from the perspective of customers. In this case a player that loses hours of progress because of a bug deserves some atention
Unless you understand the root cause of a bug there is no guarantee that an easy fix won't cause it's own problems. And if a system us so complicated that you cannot understand the root cause of a bug before doing a fix then it's very likely that the fix will have unexpected side effects. The end result is more rather than fewer bugs.
You are underestimating how seriously difficult some bugs can be to diagnose. I've seen super difficult bugs in large complex systems take over a month of dedicated effort to solve, despite the eventual fix being only a few lines long. That's the kind of bug where you take dozens of pages of notes as you puzzle through figuring it out. And this is at Google, with smart engineers. As the old saying goes, it's knowing where the problem is that's hard.
This is the antithesis of defensive programming. It's a game-breaking bug that's allowed to happen because of two things:
1. They allowed companions to take damage because they marked them with the wrong property
2. They never checked if the companions went somewhere silly to warp them back to where they should be
That's not defensive, that's like calling an API without a defensive try/catch if something goes unexpectedly wrong.
If they'd have done either of these things (and sent back error reports when it happened), they'd have caught it way sooner.
I don't write games and appreciate the interactions can be very complex. While I can understand your delight in them finding the root cause, I can't understand your support for failing to put in some defensive measures when they couldn't recreate a frequently reported bug.
> I can't understand your support for failing to put in some defensive measures when they couldn't recreate a frequently reported bug
From the tweet series: "There were one or two cases before launch where this issue seemed to happen, but no one in QA ever managed to reproduce it and despite our best efforts we couldn't learn anything concrete about it"
Assuming this is true, before launch, it was NOT a frequently reported bug.
After launch, they dove in and figured out what was happening AND what triggered it.
Your complaint appears to be that they (1) had a bug, and (2) didn't resolve boundary checking in the production code. Given the number of "fall through the map" bugs in countless games, I assume #2 is, in fact, hard. Writing non-trivial code without bugs (to avoid #1) is also hard, based on...all the smart coders I've read.
Regardless of the source of the bugs, I'm cheering that they didn't just put in defensive measures and called it a day. To the point that defensive measures can be a source of bugs, consider how many "violent vibration" bugs we've seen in, say, Skyrim, Fallout, and GTA. (While I don't play, based on clips I've seen from GTA and Red Dead, those engine(s) spam duplicate entities when the boundary overlap is detected, where Skyrim/Fallout just vibrate and rattle). I find it very plausible that attempts to "warp" characters having positional bugs without understanding the source of those bugs will end up a source of new bugs.
The "rattling" in Skyrim/FO is not specific to those engines but seems to happen in all games that use Havok (e.g. all Source based games exhibit the same behavior when something gets stuck in something else).
RAGE/GTA/RDR2 are so insanely buggy games that a game developer probably can learn nothing from them, except how to not do it, and that no matter how broken your games are, you can still earn billions.
collision detection is pretty hard to do in real time. you can't just iterate through every single entity and check if it's intersecting every single triangle in the whole world and have an acceptable frame time.
They are (a solid surface), but game physics are somewhat fuzzy due to performance requirements. Some games teleport you back on top of the ground layer if you fall too deep.
Yeah... The whole try/except "in case it throws" without an understanding of IF it should and WHY it might is a huge anti-pattern that creates more bugs down the line than it may potentially solve in the short-term.
If it shouldn't throw, don't try to catch... That is an unexpected failure that needs to be understood.
I get so irritated whenever this is brought up as an "additional security measure". Like, guys, we just found the source of the bug and put in some code to fix it. If it fails again it shows we haven't understood it completely and should take another look. Not just throw in a random try-catch where the outcome is that if it fails again it's just that some string won't be rendered in the application anymore or whatever. Could take months before anyone notices and we learn that something basic has been broken all along.
Thank you for standing up for confident programming! (We could make this a new term if we wanted to :D)
EDIT: Wait, maybe there's a better term than confident programming already... I would appreciate some input if someone sees this.
There is a third route. You can add a try catch and then throw a more detailed exception (or otherwise automatically surface the issue while not degrading the user experience) showing the context of what failed, then fo attempt to fix a poorly understood bug for which you may have an incomplete set of reproduction circumstances for.
The best solution is, as always, context and resource dependant.
When I was interning at IBM (~15 years ago now), I saw a bug report closed with "added catch for NullPointerException" as the solution. Something tells me that developer didn't really dig to the bottom of the issue...
Well it depends on where. For ex: If a request handler in your web server is throwing an exception, it's not a good design for the whole server to go down because you thought it was a bad pattern to catch any exception that handling one request might throw. The whole request should fail, return a 500 to the client, and log exception details for later examination, but shouldn't be allowed to disrupt the rest of the server.
Thing is, request handlers cross a boundary. Thus, you really should expect them to get malformed input. This means wrapping it in a try catch because 'input might be bad' is a pretty sane thing to do.
At the same time, you really should try to validate inputs. The idea being that most exceptions caught should be of the form 'Input was invalid'. Rather than 'function crashed'.
Well the parent seems to argue for at least only a very specific and limited catch. Perhaps the most common error is, say, invalid JSON. You wouldn't want to catch only the exception for that though. Maybe that's the most common, but who knows what other kind of subtle bugs might be in there. And the rule of thumb is that the web server process should never go down, no matter what kind of weird data you throw at it. That makes that indeed a case of catch everything, log it, and then move on to handle the next request.
Sorry, I probably wasn't very clear - I'm not arguing that a try/except MUST be very targeted.
> without an understanding of IF it should and WHY it might
rocqua explains it best - the API handler is an "input" to the overall system. That input may be "invalid" in that it fails to handle its own errors. In that case, you should catch a general exception rather than crashing. In this case, you know that it might raise an error, why it might raise an error, and that it's OK to ignore (well, log/alert/whatever.)
The situation I was responding to is very different - it's basically suggesting rather understand what the API you're invoking may be trying to tell you, just try/catch anything it throws and ignore it if you didn't expect it.
> Those defensive measures you describe are the kind of hacks that introduce more and more complexity.
On the other hand, a system where damage is prevented in some ways and assumed not to happen in others is already an overly complex construction of interacting systems. Switching to an invulnerability flag could simplify the system.
This can't be understated. Like, for this particular thing? Yeah, you could say "oh, they're out of range of the ship and that should never happen, warp them back." But you have to have them escape the zone first to know you should do that. And what's the most likely way to do that? Falling out of the world. Not climbing. So there's a good chance you'll miss it even if you thought about it!
This is exactly the sort of bug that anybody under the kind of bone-crunching compression you see in game development is gonna fix after they see it. "Don't start bandaging until you bleed."
I think you're right that it would have been a better design if it weren't possible for the companion to die in the first place. But I also agree with the parent that they were right to find the actual cause of their assumptions being violated before deciding how to implement the fix.
Not sure why this is getting so aggressively downvoted. A coordinate sanity check on entities is trivial to do, and at the very least should be logged.
Its not just that, but why test quest failure for if the companion is dead, if in principle companions can't die? This is a typical case of game design where lower level engine systems like physics dictate behavior/cause bugs in higher level functionality like the quest system.
Game engines do everything every frame by default, and the game has to disable/enable stuff from happening, which leads to bugs like this. With a more functional approach instead of tweaking the engine state these bugs can be avoided.
> why test quest failure for if the companion is dead, if in principle companions can't die?
The game's hardest difficulty setting - "Supernova" - allows companions to die [1] (as well as a bunch of other changes that make the game tougher, not just stronger enemies)
A very basic implementation would use a bounding box. This is done quite often in MMO type games to make sure players don't end up outside the map geometry somehow.
As I said it can be easy or hard depending on the complexity of deciding what is valid or invalid. Just because you can think of something very basic doesn’t actually make it applicable to the task at hand.
You are wrong, but not downvote worthy wrong. Lets put a artificial cube around the ship, that teleports npcs back to a safe start location (lets call it there bunk).
Now you created a whole zoo of new troubles, including a whole set of troubles you cant even predict, because you didnt know about them when you made this giant default behaviour case.
if (specific case){
}
else (everything else that could ever happen, inlcuding all future edgcases){
// here be dragons
}
Examples? Leveldesigner adds elevator, that transports player and npc ocassionally outside of the teleport box.
Skybox entitys are npcs that live outside of the box- so space battle npcs, get telported to bunks..
etc. etc.
Handling the unknown - with any other paradigm then the fail early, fail loud - is the root of evil.
This thread is speaking to how regular programing and game play programming vastly differ.
In regular programming scopes are small, small enough for programmers to accomodate all expected logic. In games interaction scope is unlimited, or atleast that is what the game designers want.
If you try to program every expected system interaction you would never finish. So instead game programming is about making systems flexible. For example, as far as the gameplay code is concerned the inside of players ship is just yet another level. The bounds of any level are defined by what the map designers have layed out, not any pre-planned "ship area".
There are no meetings where project managers request "more playable area for level 7".
Likewise the systems programmer who made the furniture system was not the script writer who created the ladders. The script writer likely thought it was brilliant and elegant to split the entrances and exits, because in game programming it was indeed brilliant. It allowed the save system's existing code to handling what would have otherwise been special handling. It allowed the existing combat logic to handling what would otherwise be special handling.
Of course regular programming is different. In regular applications if a user's data is being sent to a server there is no secondary system which needs to be allowed to kill this data. Nor is there any save system which needs to be able to save and restore an in-progress action.
Games programming is all about figuring out ways to allow more varied and more complex interactions without exploding the implementation exponentially.
I would also add to this that in my opinion often the best games (from a design perspective) are the ones that lean into this sort of freedom. This line of thinking has very much influenced game design trends, at least within certain genres. For example, simulation and survival games usually try to expose the game's systems to the player in a way that lets the player be creative in the game (Minecraft, Prison Architect, RimWorld, Factorio, etc). Even in games that aren't based on simulation or user generated content, designing the game's systems in this way gives the level creators freedom to do more without having to write code specific for every level.
Why is this a difference between "regular" programing and game play programming? Any kind of programming could have complex interactions between component. A few well known examples: OS kernel and drivers, VM manager, internet browser. The list is long, and there is a lot of programming of complicated products in the world - not everything is web development or CRUD apps.
In a game, you often want unexpected and whimsical emergent behavior. You want designers to build new and exciting things the programmer never expected. Game logic is about fun. Correctness is very low on the list of requirements.
This is not the same kind of thing as bank software.
Regular programming and game programming have different constaints and goals.
Yes regular programming could indeed add a system where numbers might be animated. During the animation maybe you want another system to be able to change the number. Maybe the number disappears, make sure you're front end can handle not having the requested data arriving. Maybe you want systems which add extra fields to user data, maybe you want anytime a user attempts login that there is a 1% chance they receive a reward determined by their "user rank".
All these are "possible" but do not happen in regular programming. If a user attempted to log into your application, and instead a window popped up saying "you have died from starvation, please recreate your account and play again" rightly so your users would be angry, not impressed.
Different constraints and goals lead to different priorities. Games on a per screen basis have a lot more budget available. Imagine if your application's login screen took 2 months to program and required the full time effort of multiple artists.
If you think about it, games are by themselves are a prominent software engineering antipattern - what they model is a huge pile of global mutable state, where everything potentially depends on everything else, and changes happen asynchronously.
Games usually have state machines as well, but you can't have a single state machine for the whole world. So it ends up being a state machine per actor, usually - and every actor can potentially interact with most other actors.
”Regular programming” and ”regular programming” have different constraints and goals. Unless you think that e.g. a sound card driver and a web browser are developed with the same goals in mind and under the same constraints, you agree that games are not exceptional in this regard.
IMO it is more useful to think of software and its development as simply more or less complex.
Also, your whole post basically ignores GP’s argument that not everything is either a CRUD app or a game. Maybe it would serve your argument better if you took care to read and respond to his.
Regular programming is not different. You mention the case of users data being sent to a server. There is so much which can go wrong there when authentication, load balancing, session management, data storage, and network latencies interact.
Much of what player consider progress in games is the industries ability to add more complexity. If you look at the contemporary equivalent to chess that would be the spin-off of MOBAs: AutoChess.
AutoChess may bare a similar name to chess but it is a vastly more complex game. In theory AutoChess could have been created on the NES, but no one would have understood how to keep the implementation reasonable.
Instead it took a progress, from Chess we got turn based strategy games. From TBS we got real time strategy games. From the early RTS games, we got the story rich Warcraft. From Warcraft came extensive modding support and story based characters. With said mod tooling and characters a series of mods became what we now know as LoL and MOBAs. Another decade and another set of mod support brought AutoChess.
Yes simulations are complex games, but what used to be a defined genre is now an aspect of most big games. Western RPGs certainly try to push the boundary of coplexity, as to tycoon or simulation games. Still, all genres are tending towards more complex systems.
While bugs in game programming are quite obscene, and the task of making a game quite arduous, all this tweet did was demonstrate how really bad code design will bite you in the ass if you're making games.
Brilliant. It shows how your code is never encapsulated enough. You think physics should apply to all agents in your game and you think climbing a ladder is trivial and then someone thinks they can easily stop all NPCs from starting new Interactions. Boom.
In that particular case, what would be your favorite fix. I tend to say that the second step of climbing a ladder should probably not count as a new interaction.
I don't know if "encapsulation" is the right word, but the "undamageable" state should definitely have included fall damage. No point in having more than one way to reduce health points, damage is damage.
I think you're giving a bit too much credit to game logic codebases.
They generally move at a pretty fast pace and change often(due to the iterative nature of what's "fun"). I don't think I saw a single unit test until I was well out of the game industry.
My go-to exception for this has always been Factorio. They have not only unit tests, but integration tests, regression tests, fuzzers, performance regression tests, ....
Their blog regularly goes into details about how it all works. It's very much worth reading.
It seems to me if the physics engine and whatnot is deterministic I don't see why it wouldn't be possible to record input sequences and keep those and their effects as regression tests.
Those tests can be written, but if you do it naively coverage is measurable and very small.
And the physics engine is not necessarily deterministic (or if it is, you can't always control all it's parameters)
It's been a few years but if I recall correctly we did have a "smoke test" running on a jenkins box under someone's desk that booted a few levels with most of the game entities(they were called danger_room_1/2/3/etc) and verified that the game didn't crash. Aside from that though there wasn't much else which lead to some spectacular build breaks.
At first I figured the physical solution, not being able to climb higher than the ladder in any case, would seem to be a reasonable one. And you probably want to do that. But that might somehow end up with characters stuck on a ladder if the dismount event never fires for some reason, so making the ladder a single event and\or exempting the dismount from the "blockable furniture actions" list, should also be done.
They should probably also consider adding fall damage as part of the considerations in invulnerable mode.
Interactions should be atomic. Unexpected outcomes in a state machine should be rejected (and probably errors logged if in dev mode).
Edit (additionally):
If they can't be made atomic, make the transitions timed, with expiration, never allow deadlocks to occur and always branch to a fail-safe state. Players will tolerate things like the uncontrollable NPC getting pinched off screen and respawning, even with a comical animation if it's that kind of game; they hate a blind referee ruining things for no perceivable reason.
I don't think I agree with this if you mean climbing ladders should be atomic... what if I save/load the game while someone is in the middle of climbing one? What if the level changes somehow unloading them? What if a level designer wants someone to start on a ladder?
In the case of Outer Worlds, I don't recall having seen any ladders where it would be unsafe to pick started or finished and snap to those states over the save.
In the case of another game where someone's actually making progress up a very large structure, I can't think of any game offhand where saving in such a state has actually been allowed. However allowing a save there by necessity triggers the edited version of my statement which includes a timed state transition and safe aborts for failures in state change. Such a save would also include the current state of progression as well as the remaining limit for the timer.
The companions in The Outer Worlds do all kinds of insane things anyway. They are always getting caught on the wrong side of doors, or outside of elevators. Or you leave them behind and they just teleport in next to you.
Probably the most ludicrous is that when sneaking, only the player is taken into consideration by enemies. I've seen patrolling enemies kick my companions out of the way without actually noticing them.
All of those are mostly because any other policy would result in companion-less runs of the game. AI just isn't there yet to read and react to the situation in a way that should take such outcomes in to consideration, and arguably that effort might never be something custom tailored to a game, it'd have to be off the shelf with maybe just some tuning and/or helper hints.
Reading the player's intent would probably still be a very difficult problem, so even with the perfect knowledge of how to interact in the world they'd probably not act and react to changes in the player's intent anywhere near as well.
Fortunately they tend to not just immediately go Leeroy Jenkins as soon as an enemy somewhere in the far distance goes active.
Skyrim was/is awful about that. I'd be sneaking along through a tomb, and then Lydia screams "I am sworn to carry your burdens!" and charges in, waking up a dozen draughir.
Billions of dollars of programming time can be spent on self driving cars, not so for your NPCs.
When large amounts of programming time is spent on game ai it does end up "self driving" at an adequate level, see Google's sc2
bot, Google's older arcade game bot's, and openai's dota bot. It's just not worth it investing that amount of time into most games.
> I tend to say that the second step of climbing a ladder should probably not count as a new interaction.
That might make sense when you view it through this specific lens, but that design decision allowed them to save character positions on ladders and make 'getting on' and 'getting off' be separate 'things' - just off the top of my head.
The '2 interactions' method actually makes the most sense in the context of their game engine - it's the same way getting 'on' or 'off' a chair works.
Their fix of just allowing interactions is probably the most correct option. That is essentially toggling a setting, as opposed to re-engineering how ladder mechanics work.
Good chance the dual "furniture" is because "sitting in furniture" triggers a separate animation cycle. Then through root-motion animation the NPCs are climbing. Getting on switches NPCs to the climb animation set. Getting off the later swaps NPCs back to regular animation set.
Altogether an elegant solution for managing what is otherwise a special case.
I'm not sure I understand why NPCs are prevented from interacting with the environment while you're talking... given that gameplay isn't stopped. Seems like an arbitrary decision that might lead to other surprising behavior too... e.g. talking to someone else to interrupt an NPC performing a scripted action.
Probably because interacting with the environment could cause environmental changes that might affect you.
For example, if you're standing in a doorway talking to someone and an NPC wants to close the door, it could close in between you and the NPC you're talking to.
I seem to remember a fallout clip where the character is trying to have a conversation with the NPC but then a monster is also attacking the NPC or something and the NPC is just oblivious. Kind of breaks the immersion when you are having a normal conversation and the NPC is taking mortal wounds at the same time.
It would be hard to make believable interactions when unexpected environmental side effects can happen at any time
My philosophy is that an important simple invariant shouldn't be maintained through the emergent behavior of a complex system.
If companions should be unkillable, then don't count on eliminating all sources of potential damage. Really make them unkillable. A ladder shouldn't ever take someone higher than the ladder goes so if some guy attempts to climb higher than the ladder goes, then freeze them in place or make them climb in place. Since these shouldn't trigger, crash the game if in dev mode, and if in prod mode check whether player allows telemetrics and if so phone home.
Those are the failsafe fixes.
Finally fix the root cause here by special casing that getting off a ladder during interaction is allowed.
Well, don't let NPCs fall to their death if they are potential companions.
These problems arise when a game engine starts as a generic physics simulation and various restrictions/rules mandated by gameplay design are added as an afterthought.
This may sound a bit silly, but depending on the story needs, these can potentially be completely separate entities. That way you wouldn't need to create a weird crossover entity that switches between friendly and enemy AI components etc.
I had the same problem in a hardware product using a legacy wired protocol that was sent over a ZigBee connection. People would complain that volume would uncontrollably go to 100% when operating a remote.
Reproducing it was hard but the culprit was WiFi interference. The protocol had separate on/off commands and no provision for lost packets since it wasn't natively designed for wireless. If the off command dropped out everything would act as if a button was still pressed.
Probably the design constraints here make this pretty necessary: you want the user to be able to hold the button down to adjust the volume by large amounts and release when it gets to a comfortable level; sending individual volume packets this quickly causes too much traffic over the slow connection; and the remote doesn't have any way to receive packets from the controlled device so it can't just send absolute volume levels.
Another constraint might be power draw. If the remote was battery operated every time you fire up the wireless bits you drain the battery some amount. More wireless time == shorter battery life.
That'a well-worth read to everyone that enjoy reading about QA finding bugs, even if not interested in games
Finding quest-related bugs is a nightmare in open-ended games...
So I work in games as well, and one thing you learn really quickly is that this "one weird bug that QA has seen once and no one has been ever able to reproduce or even come up with an idea of how it could happen" will be reported thousands of times and cause uproar on forums once you sell 10+ million copies. It's just a statistical inevitably.
Ditto in enterprise, but that doesn't mean you're ever able to reproduce it in house before it ships. You do the best you can when you have a signal to follow. And there are probably other open bugs that impact more customers that deserve higher priority anyway.
For all the flak people usually give the telemetry systems, this bug sounds like it could be made trivial if devs could push custom reporting to users. "When the companion dies, send me last few events and a minimal world-state". Having ready data collection/reporting when things go weird is super helpful, whether it's an own, server side software, or a remote client.
One reason it was so hard to pin down is that it was impossible to tell when the bug actually happened -- all of the cases we had were essentially "hey something bad happened in the last ten hours and now my quest is broken" (5/18)
You generally do not enable the level of logging that I imagine would be necessary to find this bug, on the enduser version of the software. I could be wrong.
"logging"/telemetry can be flight-data-recorder style like XRay uses [1]. This can be limited to a smaller window because it writes to a circular buffer, then again you could fit a lot of entries in a ~512MB buffer.
As another comment noted, you would log into a ring buffer in memory constantly and only dump it after the event in question happened. But without an initial state at the start od the log, this may not be enough.
They can get to a "downed" state, but they cannot die.
Depending on how early they realized it was limited to companions on the ship, with a specific state (the source is not clear on how early they determined that to be) they could log events that damage a player when in that specific state.
That would have gotten them to fall damage, at which point they might be able to make the leap to ladders. If not, they could then focus their logging around changes in elevation when in that state.
But that's backward reasoning from the already known end state, so it's not fair to apply in hindsight. It's also not clear how much log data they can ingest, whether end-user telemetry was feasible, etc...
Their non-logging-based investigation got them to the point that they were aware the problem occurred when a companion fell from a great height while the player was on their ship:
> The only place in the game when a companion is present but not in the active party is when the player is on their ship
> Eventually we figured out that "undamageable" does not mean "invulnerable" -- they can't take damage from attacks but can still get hurt from other things
> One of those things: falling a great distance
> The problem with that is that there are no spots in the player's ship that are high enough to result in a lethal fall
> So now we had to figure out how companions were mysteriously ending up way above the level
> I looked into tons of theories
This does look like logging would have caught it. They knew exactly what they were looking for: companions reaching strange heights while the player is on their ship.
The debugging is to reconcile "Observation: quests are failing because companions die" which contradicts "My understanding of the game rules: companions can't die". One of the ways the companions don't die is that the companions are revived after combat, and it's only non-companions that are 'really dead'. The other way is "could die from fall damage on the ship, but there's nothing that high".
Turns out the flaw in the logic was that the NPCs can reach great height on the ship. But the flaw in the logic just as easily could have been in the combat. (Or some other way that the tweeter didn't mention).
In any case, it's really fun to read the "aha! got it" moment.
No need for telemetry. Keep a local log (of what you would be sending via telemetry), and when the player decides to file a bug, inform them that'll include sending the log, with the option to cancel/file bug without a log.
Many bug reporting systems already work this way, such as the one used by Firefox, I think.
Just because something can be done by violating user privacy, that doesn't mean it's the only way, or even that alternatives are difficult.
Eh, that gets you things big enough that someone bothers to send a bug report. Startup performance, mis-clicks, etc, will be missed. Not to mention ragequits that don't produce a bug report...
The problem is that devs reeeeally need a mix of very specific data for debugging, and very aggregated data for prioritization... There's all kinds of cool differential privacy tools for doing things like this, but no way for users to know or trust that they do what's claimed on the tin.
Do people really get worked up over that kind of telemetry? I know I wouldn't. I would even be OK with much more telemetry around most actions in the game as long as it doesn't snoop on anything outside of what I do in the game.
I'm a fan of pure gameplay telemetry, pure error telemetry, etc... I specifically remember being amazed at the heatmaps that Bungie shared about data in Halo 2 and thinking that I needed to learn how to do that, and it was going to change how development was done forever.
But it's difficult to police good telemetry and easy to change silently. Bad actors have continually given justifiable reasons for data collection, sometimes even with a pinky-swear that they'll not use it for evil. But the terms can be updated by the party collecting the data at any time they desire, with little recourse for the user other than to stop using that product or service.
So no, I'm not worked up over that kind of telemetry, but I'm still going to block it. It doesn't make sense not to.
Yes. I don't know what is being sent back, or what information it has gathered. If a program does not require internet access for its basic functionality, and it then accesses the internet without explicitly asking me for permission, it should be viewed with extreme suspicion.
The Outer Worlds is an unexpected joy, too. Definitely an open-world game I had no idea I needed. If you haven’t, check it out. It’s basically Fallout 4 but in space.
It very much nostalgically brought me back to the beginning of the decade, when fantastic "open-world" single-player RPG's like Mass Effect 2, Skyrim, Fallout: New Vegas, Witcher 2, Dragon Age: Origins etc were being realised in quick succession. For me personally, that was a golden era. It's great to have something like The Outer Words fulfilling that itch that has been present for so long. If you like any of the games mentioned, I also recommend checking it out.
The combats quickly become too easy. The world and story are fine. I liked about New Vegas that it had a very elaborate world and the fights were more challenging.
This reads a lot like the central premise in "You" by Austin Grossman -- unkillable NPCs are occasionally found to be dead and that bug is threatening the entire game studio.
Only tangentially related, sorry: all of the the companions in The Outer Worlds were uninteresting, I found it much more satisfying (and challenging) to beat the game solo. Effectively, each companion was an ordinary human, with some arbitrary specialities, joining your crew because the current mission required it. That said, The Outer Worlds was a lot of fun (albeit short-lived).
I wholeheartedly disagree. I loved the companion missions and the dive into the characters and their psychology. They all were interesting in different ways.
Notably, I found that the Vicar had a really interesting progression as a character and his relationship with religion. It lead to some interesting philosophical dialogue, especially if you bring him when you meet the Iconoclasts. He has a healthy debate with their leader and they both come to the conclusion "let's agree to disagree". It was great.
"The game, initially developed as a joke prototype from an internal game jam and shown in an early alpha state in YouTube videos, was met with excitement and attention, prompting the studio to build the game into a releasable state while still retaining various non-breaking bugs and glitches to maintain the game's entertainment value."
It's full of awkward physics, I sort of love it. E.g. goats go up ladders by walking straight into them and continuing to walk with absolutely no other changes... so of course the body physics get a bit excited http://i.imgur.com/lNfzaDj.gif
You might enjoy speedrunning, especially older games like those for the Nintendo 64 or the original Playstation.
In addition, there's the Super Mario 64 0x A Button challenge, an effort to beat the game while avoiding the use of its main mechanic. It's not a speedrun, but the themes of exploiting every glitch possible is the same. It's also the source of the infamous "0.5x A Presses" video. https://www.youtube.com/watch?v=kpk2tdsPh0A
Several of his videos are about doing the same star with 0 A presses using a faster strat, although that might be only for the sake of the people TAS-ing the strats
You can launch yourself into the air (100s of feet), run inside a dumpster to move it, drop through the floor to start falling from the sky, booby-trap a trashcan so it flies out of no-where and hit you upside the head, walk up an invisible ramp... it's fun.
Wow that was exactly what I hoped existed! I especially liked the one where you end up beneath reality, hurtling sideways past yourself into a never-ending void.
Ending up in the ground a little bit causes a small bounce. Ending up in the ground a lot causes a large bounce. Combine that with poor collision code in tunnels...and well...
You'll probably enjoy The Stanley Parable then. If you haven't heard of it, I recommend you buy it and avoid actually reading about what it is. If you don't enjoy it I'll refund you myself ;)
Gamedev is full of interesting problems. The industry scares me because of a few friends. Fun to hack on, on the side and I’m jealous of the story and mystery this team got to chew on for a while. I know it’s hard but still jealous of these gems.
I'm sure this problem wouldn't occur if they simply used Haskell for scripting. Or modeled NPC interactions with TLA+. Can someone with a Twitter account please tell them this? Such a simple solution to all these game bugs.
why is the most upvoted thing on hackernews right now a story about a bug that came about as the result of bad code
> The only logical culprit was a bit of scripting that runs when a companion's health reaches zero: if they're in the party, it waits for combat to end and revives them; otherwise it marks them as dead "for real
The code should never have been written this way if the companion is not supposed to die. There should have been an `is_dead()` function that always returned false.
This site has declined dramatically and I think it's a result of the quality of programmers we have today vs what we had 10 years ago before the internet became mainstream.
None of these comments nor the tweets talk about how bad that code was, but instead kiss the guy's ass.
I expect to see more and more posts about games and other trivialities and less and less posts about important topics as the years fly by.
I haven't bought this game but I watched a 12 minute speedrun of it with two devs commentating and got the feeling that QA didn't perform as well as they could have on this project.
First of all the fact that the game could be finished in 12 minutes was a surprise to both devs.
And at several points during the video dev#1 expressed surprise over a shortcut the player was taking, while dev#2 said he did that in testing all the time.
Seems like a lack of both QA and communication.
Projects are often rushed to completion for shipping, and open world games can't be the easiest thing to test. You're balancing player freedom and bug hunting. It's a challenge I can only imagine.
On the other hand, if you watch speedrun records on YouTube or watch the Summoning Salt documentaries about them, expert speedruns (especially those that allow glitches) are often 10% or less the time that an ordinarily skilled player might take to finish the game. For example, the Super Mario Bros. record is under five minutes, while the Castlevania record is under 11.5 minutes.
I don't think the fact that developers are surprised by speedrun times means that a game is necessarily bad! (And even many of the most extraordinarily beloved games have glitches, often because realistic physics simulation is hard.)
I think I saw a video of Bennett Foddy in which he noted that other people could complete "Getting Over It with Bennett Foddy" considerably faster than he could, and maybe faster than he imagined anyone would be able to.
programming quality and program quality are two very separate things. a great game can have terrible programming. and an awful game can have exceptional programming.
In good software, the bugs don't cause user-facing issues. QA teams aim to make software good by finding them.
Speedrunners aren't users or QA. Speedrunners find bugs no matter what. From the developer's perspective it really doesn't matter if you can save 20 hours beating a game by clipping through the wall next to the boss, if that clip is so immensely complicated to perform that no user would ever encounter it naturally. The devs can be surprised, or not surprised, to hear their game was beaten much faster than they anticipated by speedrunners; either way it does not make their game worse or rushed or a product of bad communication.
The Legend of Zelda: The Wind Waker came out in 2002 and sold 4.6 million copies. In July of this year, a new bug was discovered by dedicated minds, not randomly but after nearly two decades of hard work, that saved two hours on the speedrun by clipping through a barrier that was supposed to be unpassable. Does it really matter from a quality standpoint if this bug exists or not? Would your original rating of The Legend of Zelda: The Wind Waker be lessened 17 years later because this bug was found by people who perhaps spent more hours trying to find it than you've spent playing video games in your entire life?
There are countless great games that have glitches that speedrunners take advantage of. The "any%" category of speed running is all about that. If you're saying that any game with glitches like that had bad QA and poor communication, then either
A: That is true for all games
or
B: You're holding everybody to too high a standard.
Some recent examples: There's a glitch in Zelda: Breath of the Wild where if you shield surf onto an enemy while it's taking damage while you're in slow-mo because of shooting an arrow while falling, you'll bounce off of them about a mile into the sky. In DOOM (2016), there's a physics bug that lets you rocket into the sky by standing on a railing and then... standing up. You can also glitch through doors by triggering a specific take-down animation on zombies.
Those are incredible games, and relatively bug-free.
Speed runs frequently rely on glitches. There was one for Oblivion where you could glitch through some geometry in the starting area and end up on the other side of the wall that was supposed to be inaccessible until the ending cut scene. The game was 'over' in like 5 minutes
I liked the glitch in Two Worlds - if you don't talk with the fist questgiver (who is a disguised end boss) and instead attack him and get him to hit some villager npc in nearby village with fireball splash damage - all the villagers attack him in revange and kill him quite quickly - and the credits roll :) It's like 3 minutes and it (unintentionally) makes a great statement - normal people don't need heroes, we just need to work as a team and we can win by ourselves ;)
My favorite game to watch speedruns of is a game called Antichamber. It's a great first person puzzle game. But it has funky rules, is non-linear and is pretty flexible. It gradually exposes you to the mechanics and gradually teaches you. I finished the game and watched a speed-run and was blown away. It mostly made sense what was done, but it was all combined in unexpected ways.
> that was our fallback, but
> -without actually knowing the cause, we couldn’t be sure that’d fix it
> -whatever the cause of the problem is might be causing OTHER problems, so we should find it
> -games near ship are fragile and it’s best to not make changes you don’t 100% understand
_THIS_ is someone that is practicing good software practices. I have no doubt this is why Outer Worlds is so much less buggy than, say, recent Bethesda _patches_. I saw a report this morning that the latest Fallout 76 patch causes some armors to degrade when a weapon reloads. This sounds exactly like the kind of bug you'd get from "fixing" something without understanding the actual cause, and thus just kicking up more emergent bugs because of your fix.
Complex interactions (like modern games), filled with shortcuts and assumptions (like modern games) means some bugs are inevitable. Requiring rigorous understanding before making changes can keep those bugs from multiplying.
...says someone with zero experience in coding games, but plenty of experience both fixing and generating bugs.
[Edit: fix formatting]
There's nothing specific about software in there. My mechanic does the same type of reasoning.
Though in defense, most of them are good enough to guess right 90% of the time. And mechanics generally get paid by the task, not hour. Replacing the probably misbehaving X pays. Spending an extra half an hour making sure that X really is broken and replacement is the best solution doesn't pay. So they just replace the X and hope for the best, and then replace something else if it didn't work.
That we do have to worry about breaking something in the software industry field, just tells us how far we've yet to go.
Take all the time you think you can afford if yyou software is not released. But it is being used by customers and such a large bug is in it and if you are not reacting to it then imo it is a problem.
I think people disagreeing with me is unable to see from the perspective of customers. In this case a player that loses hours of progress because of a bug deserves some atention
> If there was an "easy" fix that can be applied in a day or two,
> it should have been done. Agreed.
However, when someone asks for this, the implicit assumptions are that the "easy fix"
- actually fixes the symptoms (even though ignoring the root-cause),
- and does NOT introduce additional unexpected behaviour.
Ensuring this may sometimes make the easy fix just as time-consuming as a proper fix, thus favouring doing it once properly.
This is the antithesis of defensive programming. It's a game-breaking bug that's allowed to happen because of two things:
1. They allowed companions to take damage because they marked them with the wrong property
2. They never checked if the companions went somewhere silly to warp them back to where they should be
That's not defensive, that's like calling an API without a defensive try/catch if something goes unexpectedly wrong.
If they'd have done either of these things (and sent back error reports when it happened), they'd have caught it way sooner.
I don't write games and appreciate the interactions can be very complex. While I can understand your delight in them finding the root cause, I can't understand your support for failing to put in some defensive measures when they couldn't recreate a frequently reported bug.
From the tweet series: "There were one or two cases before launch where this issue seemed to happen, but no one in QA ever managed to reproduce it and despite our best efforts we couldn't learn anything concrete about it"
Assuming this is true, before launch, it was NOT a frequently reported bug.
After launch, they dove in and figured out what was happening AND what triggered it.
Your complaint appears to be that they (1) had a bug, and (2) didn't resolve boundary checking in the production code. Given the number of "fall through the map" bugs in countless games, I assume #2 is, in fact, hard. Writing non-trivial code without bugs (to avoid #1) is also hard, based on...all the smart coders I've read.
Regardless of the source of the bugs, I'm cheering that they didn't just put in defensive measures and called it a day. To the point that defensive measures can be a source of bugs, consider how many "violent vibration" bugs we've seen in, say, Skyrim, Fallout, and GTA. (While I don't play, based on clips I've seen from GTA and Red Dead, those engine(s) spam duplicate entities when the boundary overlap is detected, where Skyrim/Fallout just vibrate and rattle). I find it very plausible that attempts to "warp" characters having positional bugs without understanding the source of those bugs will end up a source of new bugs.
RAGE/GTA/RDR2 are so insanely buggy games that a game developer probably can learn nothing from them, except how to not do it, and that no matter how broken your games are, you can still earn billions.
Can’t gamedevs reinvent solidness and make underground/walls solid? Viscosity/density/archimedes?
Defensive programming is not fixing bugs with more features, quite the opposite: it is failing hard when a condition does not hold.
If it shouldn't throw, don't try to catch... That is an unexpected failure that needs to be understood.
Thank you for standing up for confident programming! (We could make this a new term if we wanted to :D)
EDIT: Wait, maybe there's a better term than confident programming already... I would appreciate some input if someone sees this.
The best solution is, as always, context and resource dependant.
At the same time, you really should try to validate inputs. The idea being that most exceptions caught should be of the form 'Input was invalid'. Rather than 'function crashed'.
> without an understanding of IF it should and WHY it might
rocqua explains it best - the API handler is an "input" to the overall system. That input may be "invalid" in that it fails to handle its own errors. In that case, you should catch a general exception rather than crashing. In this case, you know that it might raise an error, why it might raise an error, and that it's OK to ignore (well, log/alert/whatever.)
The situation I was responding to is very different - it's basically suggesting rather understand what the API you're invoking may be trying to tell you, just try/catch anything it throws and ignore it if you didn't expect it.
On the other hand, a system where damage is prevented in some ways and assumed not to happen in others is already an overly complex construction of interacting systems. Switching to an invulnerability flag could simplify the system.
In a modern open-worldish game, this is pretty nontrivial.
This is exactly the sort of bug that anybody under the kind of bone-crunching compression you see in game development is gonna fix after they see it. "Don't start bandaging until you bleed."
Game engines do everything every frame by default, and the game has to disable/enable stuff from happening, which leads to bugs like this. With a more functional approach instead of tweaking the engine state these bugs can be avoided.
The game's hardest difficulty setting - "Supernova" - allows companions to die [1] (as well as a bunch of other changes that make the game tougher, not just stronger enemies)
[1] https://www.pcgamer.com/uk/which-the-outer-worlds-difficulty...
Now you created a whole zoo of new troubles, including a whole set of troubles you cant even predict, because you didnt know about them when you made this giant default behaviour case.
if (specific case){
} else (everything else that could ever happen, inlcuding all future edgcases){
// here be dragons
}
Examples? Leveldesigner adds elevator, that transports player and npc ocassionally outside of the teleport box.
Skybox entitys are npcs that live outside of the box- so space battle npcs, get telported to bunks..
etc. etc.
Handling the unknown - with any other paradigm then the fail early, fail loud - is the root of evil.
In regular programming scopes are small, small enough for programmers to accomodate all expected logic. In games interaction scope is unlimited, or atleast that is what the game designers want.
If you try to program every expected system interaction you would never finish. So instead game programming is about making systems flexible. For example, as far as the gameplay code is concerned the inside of players ship is just yet another level. The bounds of any level are defined by what the map designers have layed out, not any pre-planned "ship area".
There are no meetings where project managers request "more playable area for level 7".
Likewise the systems programmer who made the furniture system was not the script writer who created the ladders. The script writer likely thought it was brilliant and elegant to split the entrances and exits, because in game programming it was indeed brilliant. It allowed the save system's existing code to handling what would have otherwise been special handling. It allowed the existing combat logic to handling what would otherwise be special handling.
Of course regular programming is different. In regular applications if a user's data is being sent to a server there is no secondary system which needs to be allowed to kill this data. Nor is there any save system which needs to be able to save and restore an in-progress action.
Games programming is all about figuring out ways to allow more varied and more complex interactions without exploding the implementation exponentially.
Originally, when I read the article, I thought that the issues described may be a result of bad software design.
But maybe it’s because applications are about strictness, and games are about freedom.
In application programming, you should be in a defined state most of the time, and make state transitions short and atomic.
In guess that in games, you are really in one huge, never-ending state transition.
This is not the same kind of thing as bank software.
Yes regular programming could indeed add a system where numbers might be animated. During the animation maybe you want another system to be able to change the number. Maybe the number disappears, make sure you're front end can handle not having the requested data arriving. Maybe you want systems which add extra fields to user data, maybe you want anytime a user attempts login that there is a 1% chance they receive a reward determined by their "user rank".
All these are "possible" but do not happen in regular programming. If a user attempted to log into your application, and instead a window popped up saying "you have died from starvation, please recreate your account and play again" rightly so your users would be angry, not impressed.
Different constraints and goals lead to different priorities. Games on a per screen basis have a lot more budget available. Imagine if your application's login screen took 2 months to program and required the full time effort of multiple artists.
It’s very hard especially if functionality gets added mid process (the companion can die now).
I had a piece of code log (“you should never get here”) once before crashing. Some programmer long gone put a constraint check with that strange note.
It’s hard especially if functionality gets added mid process (the companion can die now).
I had a piece of code log (“you should never get here”) once before crashing. Some programmer long gone put a constraint check with that strange note.
IMO it is more useful to think of software and its development as simply more or less complex.
Also, your whole post basically ignores GP’s argument that not everything is either a CRUD app or a game. Maybe it would serve your argument better if you took care to read and respond to his.
The type of programs where things like this occur are called simulations.
Much of what player consider progress in games is the industries ability to add more complexity. If you look at the contemporary equivalent to chess that would be the spin-off of MOBAs: AutoChess.
AutoChess may bare a similar name to chess but it is a vastly more complex game. In theory AutoChess could have been created on the NES, but no one would have understood how to keep the implementation reasonable.
Instead it took a progress, from Chess we got turn based strategy games. From TBS we got real time strategy games. From the early RTS games, we got the story rich Warcraft. From Warcraft came extensive modding support and story based characters. With said mod tooling and characters a series of mods became what we now know as LoL and MOBAs. Another decade and another set of mod support brought AutoChess.
Yes simulations are complex games, but what used to be a defined genre is now an aspect of most big games. Western RPGs certainly try to push the boundary of coplexity, as to tycoon or simulation games. Still, all genres are tending towards more complex systems.
In that particular case, what would be your favorite fix. I tend to say that the second step of climbing a ladder should probably not count as a new interaction.
They generally move at a pretty fast pace and change often(due to the iterative nature of what's "fun"). I don't think I saw a single unit test until I was well out of the game industry.
Their blog regularly goes into details about how it all works. It's very much worth reading.
It's been a few years but if I recall correctly we did have a "smoke test" running on a jenkins box under someone's desk that booted a few levels with most of the game entities(they were called danger_room_1/2/3/etc) and verified that the game didn't crash. Aside from that though there wasn't much else which lead to some spectacular build breaks.
Ah the joys of "AAA" development.
They should probably also consider adding fall damage as part of the considerations in invulnerable mode.
Edit (additionally):
If they can't be made atomic, make the transitions timed, with expiration, never allow deadlocks to occur and always branch to a fail-safe state. Players will tolerate things like the uncontrollable NPC getting pinched off screen and respawning, even with a comical animation if it's that kind of game; they hate a blind referee ruining things for no perceivable reason.
In the case of another game where someone's actually making progress up a very large structure, I can't think of any game offhand where saving in such a state has actually been allowed. However allowing a save there by necessity triggers the edited version of my statement which includes a timed state transition and safe aborts for failures in state change. Such a save would also include the current state of progression as well as the remaining limit for the timer.
Probably the most ludicrous is that when sneaking, only the player is taken into consideration by enemies. I've seen patrolling enemies kick my companions out of the way without actually noticing them.
Reading the player's intent would probably still be a very difficult problem, so even with the perfect knowledge of how to interact in the world they'd probably not act and react to changes in the player's intent anywhere near as well.
Skyrim was/is awful about that. I'd be sneaking along through a tomb, and then Lydia screams "I am sworn to carry your burdens!" and charges in, waking up a dozen draughir.
When large amounts of programming time is spent on game ai it does end up "self driving" at an adequate level, see Google's sc2 bot, Google's older arcade game bot's, and openai's dota bot. It's just not worth it investing that amount of time into most games.
That might make sense when you view it through this specific lens, but that design decision allowed them to save character positions on ladders and make 'getting on' and 'getting off' be separate 'things' - just off the top of my head.
The '2 interactions' method actually makes the most sense in the context of their game engine - it's the same way getting 'on' or 'off' a chair works.
Their fix of just allowing interactions is probably the most correct option. That is essentially toggling a setting, as opposed to re-engineering how ladder mechanics work.
Games are complicated.
Altogether an elegant solution for managing what is otherwise a special case.
For example, if you're standing in a doorway talking to someone and an NPC wants to close the door, it could close in between you and the NPC you're talking to.
It would be hard to make believable interactions when unexpected environmental side effects can happen at any time
If companions should be unkillable, then don't count on eliminating all sources of potential damage. Really make them unkillable. A ladder shouldn't ever take someone higher than the ladder goes so if some guy attempts to climb higher than the ladder goes, then freeze them in place or make them climb in place. Since these shouldn't trigger, crash the game if in dev mode, and if in prod mode check whether player allows telemetrics and if so phone home.
Those are the failsafe fixes.
Finally fix the root cause here by special casing that getting off a ladder during interaction is allowed.
Not letting companions fall to their death, ever... Bethesda hasn't fixed these bugs in 5 years.
What would you do instead?
These problems arise when a game engine starts as a generic physics simulation and various restrictions/rules mandated by gameplay design are added as an afterthought.
Reproducing it was hard but the culprit was WiFi interference. The protocol had separate on/off commands and no provision for lost packets since it wasn't natively designed for wireless. If the off command dropped out everything would act as if a button was still pressed.
>_<
> I had the same problem in a hardware product using a legacy wired protocol that was sent over a ZigBee connection.
Just speculating though....
One reason it was so hard to pin down is that it was impossible to tell when the bug actually happened -- all of the cases we had were essentially "hey something bad happened in the last ten hours and now my quest is broken" (5/18)
[1] https://llvm.org/docs/XRay.html#flight-data-recorder-mode
Depending on how early they realized it was limited to companions on the ship, with a specific state (the source is not clear on how early they determined that to be) they could log events that damage a player when in that specific state.
That would have gotten them to fall damage, at which point they might be able to make the leap to ladders. If not, they could then focus their logging around changes in elevation when in that state.
But that's backward reasoning from the already known end state, so it's not fair to apply in hindsight. It's also not clear how much log data they can ingest, whether end-user telemetry was feasible, etc...
> The only place in the game when a companion is present but not in the active party is when the player is on their ship
> Eventually we figured out that "undamageable" does not mean "invulnerable" -- they can't take damage from attacks but can still get hurt from other things
> One of those things: falling a great distance
> The problem with that is that there are no spots in the player's ship that are high enough to result in a lethal fall
> So now we had to figure out how companions were mysteriously ending up way above the level
> I looked into tons of theories
This does look like logging would have caught it. They knew exactly what they were looking for: companions reaching strange heights while the player is on their ship.
Turns out the flaw in the logic was that the NPCs can reach great height on the ship. But the flaw in the logic just as easily could have been in the combat. (Or some other way that the tweeter didn't mention).
In any case, it's really fun to read the "aha! got it" moment.
Many bug reporting systems already work this way, such as the one used by Firefox, I think.
Just because something can be done by violating user privacy, that doesn't mean it's the only way, or even that alternatives are difficult.
The problem is that devs reeeeally need a mix of very specific data for debugging, and very aggregated data for prioritization... There's all kinds of cool differential privacy tools for doing things like this, but no way for users to know or trust that they do what's claimed on the tin.
I've been floating the idea of setting up some log shipper plugins for Unity/Unreal to make a turnkey solution you could drop in and self host.
But it's difficult to police good telemetry and easy to change silently. Bad actors have continually given justifiable reasons for data collection, sometimes even with a pinky-swear that they'll not use it for evil. But the terms can be updated by the party collecting the data at any time they desire, with little recourse for the user other than to stop using that product or service.
So no, I'm not worked up over that kind of telemetry, but I'm still going to block it. It doesn't make sense not to.
If it were opt-in, it could be used well however, similar to what the Telltale games had done.
65% of people made choice A. 35% of people made choice B.
They have a sort of proxy for that with Achievements however.
I will say that TOW was very stable for an Obsidian Game. I don't think I've experienced a bug yet.
The Outer Worlds is an unexpected joy, too. Definitely an open-world game I had no idea I needed. If you haven’t, check it out. It’s basically Fallout 4 but in space.
Notably, I found that the Vicar had a really interesting progression as a character and his relationship with religion. It lead to some interesting philosophical dialogue, especially if you bring him when you meet the Iconoclasts. He has a healthy debate with their leader and they both come to the conclusion "let's agree to disagree". It was great.
"The game, initially developed as a joke prototype from an internal game jam and shown in an early alpha state in YouTube videos, was met with excitement and attention, prompting the studio to build the game into a releasable state while still retaining various non-breaking bugs and glitches to maintain the game's entertainment value."
In addition, there's the Super Mario 64 0x A Button challenge, an effort to beat the game while avoiding the use of its main mechanic. It's not a speedrun, but the themes of exploiting every glitch possible is the same. It's also the source of the infamous "0.5x A Presses" video. https://www.youtube.com/watch?v=kpk2tdsPh0A
You can launch yourself into the air (100s of feet), run inside a dumpster to move it, drop through the floor to start falling from the sky, booby-trap a trashcan so it flies out of no-where and hit you upside the head, walk up an invisible ramp... it's fun.
https://www.youtube.com/watch?v=FMp-pzlwzlE
Also this sub that highlights small, easy to overlook game details: https://www.reddit.com/r/GamingDetails/
https://kotaku.com/guy-beats-fallout-4-without-killing-anyon...
> The only logical culprit was a bit of scripting that runs when a companion's health reaches zero: if they're in the party, it waits for combat to end and revives them; otherwise it marks them as dead "for real
The code should never have been written this way if the companion is not supposed to die. There should have been an `is_dead()` function that always returned false.
This site has declined dramatically and I think it's a result of the quality of programmers we have today vs what we had 10 years ago before the internet became mainstream.
None of these comments nor the tweets talk about how bad that code was, but instead kiss the guy's ass.
I expect to see more and more posts about games and other trivialities and less and less posts about important topics as the years fly by.
First of all the fact that the game could be finished in 12 minutes was a surprise to both devs.
And at several points during the video dev#1 expressed surprise over a shortcut the player was taking, while dev#2 said he did that in testing all the time.
Seems like a lack of both QA and communication.
Projects are often rushed to completion for shipping, and open world games can't be the easiest thing to test. You're balancing player freedom and bug hunting. It's a challenge I can only imagine.
I don't think the fact that developers are surprised by speedrun times means that a game is necessarily bad! (And even many of the most extraordinarily beloved games have glitches, often because realistic physics simulation is hard.)
I think I saw a video of Bennett Foddy in which he noted that other people could complete "Getting Over It with Bennett Foddy" considerably faster than he could, and maybe faster than he imagined anyone would be able to.
In good software, the bugs don't cause user-facing issues. QA teams aim to make software good by finding them.
Speedrunners aren't users or QA. Speedrunners find bugs no matter what. From the developer's perspective it really doesn't matter if you can save 20 hours beating a game by clipping through the wall next to the boss, if that clip is so immensely complicated to perform that no user would ever encounter it naturally. The devs can be surprised, or not surprised, to hear their game was beaten much faster than they anticipated by speedrunners; either way it does not make their game worse or rushed or a product of bad communication.
The Legend of Zelda: The Wind Waker came out in 2002 and sold 4.6 million copies. In July of this year, a new bug was discovered by dedicated minds, not randomly but after nearly two decades of hard work, that saved two hours on the speedrun by clipping through a barrier that was supposed to be unpassable. Does it really matter from a quality standpoint if this bug exists or not? Would your original rating of The Legend of Zelda: The Wind Waker be lessened 17 years later because this bug was found by people who perhaps spent more hours trying to find it than you've spent playing video games in your entire life?
A: That is true for all games
or
B: You're holding everybody to too high a standard.
Some recent examples: There's a glitch in Zelda: Breath of the Wild where if you shield surf onto an enemy while it's taking damage while you're in slow-mo because of shooting an arrow while falling, you'll bounce off of them about a mile into the sky. In DOOM (2016), there's a physics bug that lets you rocket into the sky by standing on a railing and then... standing up. You can also glitch through doors by triggering a specific take-down animation on zombies.
Those are incredible games, and relatively bug-free.
The game can be a 30 hour game if you hussle or as far as 200 if you don't.