Copying objects in JavaScript

(smalldata.tech)

124 points | by wheresvic1 1961 days ago

22 comments

  • mrgalaxy 1961 days ago
    I've been programming JS for a very long time and have learned to just stop trying to do a generic deep copy. Since JS is a dynamically typed language, it will always lead to issues down the road. Instead I write domain specific merge methods for whatever objects I'm merging.

        function mergeOptions(...options) {
          const result = {};
    
          for (const opt of options) {
            result = {
              ...result,
              ...opt
              arrayValue: [
                ...(result.arrayValue || []),
                ...(opt.arrayValue || [])
              ],
              deepObject: {
                ...result.deepObject,
                ...opt.deepObject
              }
            };
          }
    
          return result;
        }
    
    Know the shape of your objects and merging deeply becomes painless and won't have edge-cases.
    • ben509 1961 days ago
      Another way of looking at it: if you are frequently doing complex copies, you probably want immutable types.
      • throwaway645738 1961 days ago
        Im away from my usual PCs, and Im here just to say when I realized this exact same thing it all clicked for me on why to use immutable objects
    • cageface 1961 days ago
      Immerjs is a very handy library for doing this kind of thing. It is a natural fit for react but can be used for any kind of copying like this.
      • 0xFACEFEED 1960 days ago
        +1 for immer. I love it.

        If anyone here ends up using it, make sure you learn how it works first. There are performance implications. It may not matter but it's important to know what they are. Also it's generally best practice to learn how magical utilities like immer do what they do before using them!

    • lxe 1961 days ago
      I think it's not the best idea to keeping the shapes of objects in your head and manually cloning/merging them.

      This will lead to bugs, as inadvertently as a human you'll miss a merge or a clone and retain references that you don't want.

      Inability of a language or runtime to correctly and quickly clone a structure is an upsetting fact of JavaScript.

  • Null-Set 1961 days ago
    As of version 8.0.0 node has exposed a serialization api which is compatible with structured clone. https://nodejs.org/api/v8.html#v8_serialization_api

        const v8 = require('v8');
        const buf = v8.serialize({a: 'foo', b: new Date()});
        const cloned = v8.deserialize(buf);
        cloned.b.getMonth();
    • devoply 1961 days ago
      Have we learned nothing from Java's serialization fiasco?
      • foota 1961 days ago
        I don't know how the JavaScript proposal does it, but you can certainly create generic clone structures that are safe for untrusted input.
      • nur0n 1961 days ago
        I want to learn, can you elaborate?
    • wheresvic1 1960 days ago
      That's awesome, I'll update the article to reflect this!
  • malcolmwhite 1961 days ago
    Using structured cloning for deep copies is clever, but may or may not give you the behavior you want for SharedArrayBuffers. The copied value would be a new SAB with the same underlying data buffer, so that changes to one value will be visible to the other. That's good for most uses of structured cloning, but it's not what I would expect from a deep copy.

    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

    • Null-Set 1961 days ago
      The point of structured clone was originally for passing data to web workers. Since the point of shared array buffers is to share data with workers, it makes sense that the structured clone algorithm keeps the SAB identity.
    • bcoates 1961 days ago
      Programs that use SharedArrayBuffers are already defective, so it's no big deal.
      • yzmtf2008 1961 days ago
        Not true. Chrome has enabled SharedArrayBuffers again as of Chrome 68.
        • bzbarsky 1961 days ago
          Desktop-only, not on Android, right?

          Also, a program that only works in one browser, and only due to that browser being willing to open security holes other browsers are not willing to open, is arguably still defective. That said, that might describe a lot of things people are doing nowadays (e.g. anything that uses WebUSB).

  • bcoates 1961 days ago
    Fundamentaly, deep-copy in Javascript is a typed operation and no "generic" deep-copy algorithm is possible--you have to know what meaning the value is supposed to have to copy it.

    There's nothing inherently wrong with structured clone, but it's only for JSON-able objects with extensions for circular references and some built in value-like classes. It's also special-cased for safe transmission between Javascript domains so it has a bunch of undesirable behavior for local copies. (no dom, no functions, no symbols, no properties...)

    Even primitive types can't be safely copied unless you know what they're going to be used for later, a nodejs file descriptor is just an integer but it's also a reference to an OS resource that can't be duplicated without a syscall.

    • amelius 1961 days ago
      > Fundamentaly, deep-copy in Javascript is a typed operation and no "generic" deep-copy algorithm is possible--you have to know what meaning the value is supposed to have to copy it.

      Why? I see no problem if the deep-copy behaves exactly the same as the original, from the perspective of any operation in the Javascript API (except for the === operator).

      • bcoates 1961 days ago
        Exactly the same as the original is a shallow copy.

        Deep copy roughly means "If I do `dst = deepcopy(src)`, modifying anything in the world I can reach through a reference from dst should have no side effect visible through src" which is a reasonable thing to ask in some special cases (a tree of plain old javascript values, a DOM node) but not reasonable in others (a database connection object, a file descriptor, a user-id)

        Like, what should a deep copy function do if it reaches a reference to the global object? or a function that closes over some references in the object being copied?

        The answer is always "it depends on what the object will be used for later and what specific behavior you want"

      • kccqzy 1961 days ago
        The === operator is a coercion-free equality operator. It's not an object identity comparison operator.
        • XCSme 1961 days ago
          But for `Objects` (not primitves) it is the identity comparison operator, right?
          • kccqzy 1961 days ago
            I wasn't aware of that. I thought ES2015 added Object.is() for this purpose: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
            • AgentME 1961 days ago
              The main difference between === and Object.is() is that === treats +0 and -0 as equivalent, despite the fact that some math operations treat them differently. I think they added Object.is() because it was awkwardly difficult to make code tell the difference between +0 and -0.
          • ZenPsycho 1961 days ago
            technically for primatives too, since they are interned, immutable and their value is their identity?
          • scottmf 1961 days ago
            Yes
  • dan-robertson 1961 days ago
    It is fundamentally very hard (impossible?) to deep copy everything in JavaScript. References cannot be escaped because they may be hidden in non-introspectable (cruicially, non cloneabel) places. Viz:

      function f(){var x = 0; return function(){return x++;};}
      var x = {foo:f()};
      print(x.foo());
      var y = {foo:f()};
      print(y.foo());
      var z = someDeepCopy(y);
      print(z.foo());
      print(x.foo());
      print(y.foo());
      print(z.foo());
    
    If a copy were sufficiently deep then one could expect:

      0
      0
      1
      1
      1
      2
    
    However if it were not deep one would get:

      0
      0
      1
      1
      2
      3
    
    Even if one allows a deep copying of closures then this still might not work as an object which contains two (potentially different) functions closing over the same binding (ie particular instance of a particular variable) may be copied into two functions each closing over their own separate binding.

    I think the only good solution to this is to either give up trying to do deep copies or give up immutability and stop caring about deep copies.

  • russellbeattie 1961 days ago
    The language really should have a true immutable type (without freezing, etc.) and deep copy method built in, with as many caveats and parameters as needed. Coroutines would be awesome as well. (Yes, I'm thinking, "How could JavaScript be more like Go or Erlang?")

    And then it needs to stop adding new features for at least a couple years so the world can catch up.

    • yuchi 1961 days ago
      I infer you don't mean to have both immutable data structures and deep copy as features to use together, since immutables don't need to be cloned.
    • TheAceOfHearts 1961 days ago
      You can implement coroutines using generators. Is there any feature you'd miss from other implementations if you used generators in that way?
    • SonicSoul 1961 days ago
      i guess most languages do not add this because of circular reference problem?
  • nobody271 1961 days ago
    var copy = JSON.parse(JSON.stringify(myObj));

    Anything beyond this and you are begging for trouble because there's always context-specific gotchas.

    • pnevares 1961 days ago
      This is presented in the linked document with the following context-specific gotchas:

      > Unfortunately, this method only works when the source object contains serializable value types and does not have any circular references. An example of a non-serializable value type is the Date object - it is printed in a non ISO-standard format and cannot be parsed back to its original value :(.

      • simlevesque 1961 days ago
        That's false. If you use JSON.stringify on an object, it will call the method toJSON of each values recursively. The toJSON method of a Date returns the ISO string.

        source: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

        • negativegate 1961 days ago
          But JSON.parse will leave it as a string instead of converting it back to a date, so this cloning approach doesn't work for objects containing dates.
          • simlevesque 1961 days ago
            I did not say that JSON.parse would parse the dates.

            I'm saying that the article is wrong when it says that JSON.stringify does not transform Dates into ISO string.

            Right after that, the article says "cannot be parsed back to its original value" when it definitely can be parsed back. JSON.parse does not do it by default but that was never his point.

            • twblalock 1961 days ago
              > Right after that, the article says "cannot be parsed back to its original value" when it definitely can be parsed back. JSON.parse does not do it by default but that was never his point.

              The broader point, which is entirely correct, is that you don't get back an exact copy of the object you want to clone, because the date fields don't end up being the same type.

        • freeopinion 1961 days ago
          JSON.parse of an ISO string yields a string, not a date.

          let d = new Date(); let s = JSON.parse(JSON.stringify(d));

          s will not be a clone of d.

  • austincheney 1961 days ago
    Why? Why would people want to clone objects? When I have encountered this in the past it is from people who are new to the language.

    My advise to any person who really believes they need a clone of an object: do some self-reflection on plan as to why you think you need a cloned object. Any other approach is more efficient and more simple in the code.

    Objects are hash maps that store data.

  • sebringj 1961 days ago
    Yah that's why I use the serializable override:

    someObj.toJSON = function() { return { foo: this.foo, bar: this.bar } }

    It is an extra step but when using redux or something like that you have to serialize stuff anyway to store it and is especially useful for mobile react-native stuff in keeping state when phone restarts or connection fails.

  • Scarbutt 1961 days ago
    Another option if you can afford it is to just use immutablejs.

    Or make your functions return new objects.

    • realharo 1961 days ago
      I would recommend immer (https://github.com/mweststrate/immer) instead of ImmutableJS. You can work with regular JS objects, plus it plays much nicer with TypeScript.
      • benvan 1961 days ago
        I wrote a tiny library called fn-update for this that does composable functional updates (https://github.com/benvan/fn-update).

        Personally, I prefer not having to mutate data, even wrapped in a produce method

        • realharo 1961 days ago
          Having to specify the path using an array of strings would be an instant dealbreaker for me. It messes with all tooling and is bad for readability.
    • nine_k 1961 days ago
      I suppose immutable.js supports partial reuse of objects, that is, if you only change the value of one attribute, the changed copy has this attribute set differently, but the rest is shallow-copied?

      If so, indeed immutable objects would not run into the problem of copying, as long as you can afford them to be immutable. (That is, you're not working with any APIs that assume and use mutability.)

    • TheAceOfHearts 1961 days ago
      ImmutableJS is a pretty huge library. I'd conjecture most web apps don't have a legitimate need for something so comprehensive and would be better off with a simpler solution.

      If you know the shapes of your objects ahead of time you can create one-off functions, which will probably be faster and require far less code.

    • beaconstudios 1961 days ago
      I cannot recommend ramda enough. It provides the immutability and flexibility of immutablejs, but because it's build in a functional paradigm, the logic is fully composeable so complex, deeply nested changes are very simple and readable.
      • oweiler 1961 days ago
        That is an apples to oranges comparison. ramda does not provide persistent data structures.
        • beaconstudios 1961 days ago
          both are used to approach the same problem: immutable manipulation of javascript data. The fact that immutableJS provides a persistent object is an implementation detail in the strategy used to solve that problem.
  • ben509 1961 days ago
    Granted, this is Python, but I wrote this a while back: https://github.com/scooby/pyrsistent-mutable

    Basically, it's an AST translator that lets you use imperative syntax against immutable types. That is, `x.a = b` becomes the clunky `x = x.set('a', b)`, and it really gets convenient when you have complex structures.

    Would it be worth it to look into a babel plugin for Javascript and ImmutableJS?

  • chrisseaton 1961 days ago
    > objects in Javascript are simply references to a location in memory

    No variables are simply references to objects. Objects aren't references - they're referents.

  • barrystaes 1961 days ago
    I like the shallow copy, and never needed a deep copy. I am using JS for a few years tops, mostly React.

    To me its exactly what native languages do with pointers. In some languages (like Delphi) its implicit (like JS) and some (like C) its explicit in syntax.

  • KaoruAoiShiho 1961 days ago
    Been using this for a while: https://stackoverflow.com/a/44612374/663447 Best clone imo (for the correct usecases).
    • simlevesque 1961 days ago
      Don't use it with dates, it breaks them. You'll lose the data.

      # var a = new Date();

      # cloneDeep({ a }).a === { a }.a;

      This returns false. Use JSON.stringify if you care about the content. cloneDeep might be useful if you don't care about data integrity.

      • freeopinion 1961 days ago
        JSON.stringify also breaks dates.
        • simlevesque 1961 days ago
          No it does not. It returns a ISO string. You don't lose any info. And you can parse it back with JSON.parse if you give it a reviver function. [1]

          Do you want JSON.stringify to keep dates intact ? The whole point is to turn it to a string.

          [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

          • kbenson 1961 days ago
            "It works if you provide your own additional code that loops over and fixes known problems" is not the same as "it works". It's useful, but at most it saves you a few lines of boilerplate code (and perhaps a few negligible cycles) to write your own recurser to do the same thing immediately after the transform.

            > Do you want JSON.stringify to keep dates intact ? The whole point is to turn it to a string.

            There are ways to serialize data structures to strings. JSON doesn't generally do that, and it's been to its benefit as it helps keep it generic and easily usable from many languages.

            That said, there are serialization tools which can correctly serialize and unserialize more complex structures, given the correct circumstances (only core constructs, or the objects in question have serialization helpers, or they are ensured to not have features that might cause problems such as references to outside data).

            The point of this discussion is Javascript object cloning, not turning objects into a string. Offering up JSON and then responding to a criticism of it based on its methods as "the whole point is to turn it into a string" is somewhat ridiculous given many of the other options you're espousing JSON over don't use strings at all.

          • Prefinem 1961 days ago
            If you are writing a reviver, what difference is it to instead write a deep clone function?

            JSON.parse(JSON.stringify(obj)) !== deepClone(obj)

            EDIT: Also, how can you determine if the string should be a date, or a string? Sure, if it fits, you can always convert it to a Date, but if your first object is coming from something that sends ISO Dates as strings and you used JSON.parse with a reviver, you would get a different object.

          • freeopinion 1961 days ago
            The whole point is to clone an object. You lost track of the whole point.
    • freeopinion 1961 days ago
      I followed your link, which has links to two perf sites, which show that Object.assign is nearly 20x more performant than your preferred solution.
      • KaoruAoiShiho 1961 days ago
        Object.assign doesn't actually clone anything.
  • kalmi10 1961 days ago
    Fun fact: Not even the whole of number type is safe to clone with the JSON method, because Infinity or NaN turn into null.

    So one can’t infer JSON-clonability from TypeScript/JavaScript types. Learned this the hard way.

  • z3t4 1961 days ago
    Premature optimizations is the root of all evil. That said, creating new objects do slow your code down, so do it only when you have a good reason to.
  • iLemming 1961 days ago
    Every single time I see a similar article on Javascript, I feel so lucky for being able to use Clojurescript instead. Seriously - Javascript platform is great. The language itself? Not so nice. Clojurescript makes so many things simply better.
  • stevebmark 1961 days ago
    Never do any of this. It's not 1990 anymore, we don't copy code from random blog posts to solve problems.
  • freeopinion 1961 days ago
    > x = 4

    4

    > y = new (x.constructor)(x)

    [Number: 4]

    > x.constructor

    [Function: Number]

    > y.constructor

    [Function: Number]

    > typeof x

    'number'

    > typeof y

    'object'

    > x

    4

    > y

    [Number: 4]

    Is y a clone of x?

    • snek 1961 days ago
      no its boxed. take `y.valueOf()` and you've got a successful clone.
  • jackconnor 1961 days ago
    One of the weird, interesting parts of JS. Great article.
  • baybal2 1961 days ago
    20 years on, and still no native object copy and merge in JS
    • oweiler 1961 days ago
      Isn't Object.assign basically a merge?
      • RussianCow 1961 days ago
        I think the parent meant a deep copy/merge—Object.assign only does a shallow merge, which is the point of the article.
        • s_ngularity 1961 days ago
          What language does have a deep copy/merge operation built into the language?
          • chrismorgan 1961 days ago
            Rust has a Clone trait which provides the .clone() method, which isn’t a shallow clone or a deep clone, due to Rust’s ownership model—the terms “shallow clone” and “deep clone” actually don’t make sense in Rust.

            For completeness, I must mention that when you get to types like Rc<T>, deep cloning becomes a meaningful operation, but even there it’s not exposed as a deep clone method, but rather via make_mut (https://doc.rust-lang.org/std/rc/struct.Rc.html#method.make_...) which skips the deep part of cloning if there are no other references to the inner value.

            Rust’s ownership model is absolutely delightful to work with. I miss it all the time when working in Python or JavaScript, and encounter and write bugs that would have been structurally impossible in Rust from time to time—to say nothing of inefficiencies that would have been either inexpressible or unreasonable in Rust.

          • rcfox 1961 days ago
          • yoklov 1961 days ago
            Rust's `Clone` trait is deep by default, except for

            - Types with shared semantics (e.g. Rc<T>, Arc<T>), or holding onto these internally

            - Types holding borrowed references (these will still reference the same data).

            C++'s copy constructors are too, but there are more caveats, although they're similar in principal to Rust's caveats.

          • Scarbutt 1961 days ago
            Most(all?) functional programming languages.
            • RussianCow 1961 days ago
              Most functional programming languages don't make a distinction between values and references, so this is kind of a moot point since the copying/merging is never exposed to the developer.
    • zbentley 1961 days ago
      47+ years, and still no native file-descriptor copy in UNIX/C.
  • simlevesque 1961 days ago
    That article is wrong. A date can definitely be serialized in JS. It is in fact converted to a ISO string when it is transformed into JSON, but the article says it does not. The author needs to learn about JSON.stringify and the toJSON method of built-ins.
    • wheresvic1 1961 days ago
      Aha yes you are correct - it outputs an ISO string at the very least but does not parse it back to a date. I will update the article to reflect this!
      • simlevesque 1961 days ago
        Ok but please read carefuly because JSON.parse can in fact recover dates. The JSON.parse function accepts a 'reviver' function as an argument to do just that. 99% of the time it's not useful so there is no reason for JSON.parse to do it by default but JSON.parse definitely can parse dates back if you use all it's features.

        I like that it does not do it by default but I can make it parse the dates if I want.

        • Spivak 1961 days ago
          Saying that JSON.parse can do something when it's really just you implementing the transformation from string to date object is a little misleading.

          You can make JSON parse arbitrary sublanguages if you're willing to put in the work.

        • switch007 1961 days ago
          And what do you pass as the reviver argument? Your own function to check a string to see if it's valid IS8601 and parse it? That's just using parse()'s recursive walking of the object: it does no date reviving itself, right?
        • dzek69 1961 days ago
          That way you can get different result, as you could actually have date object and date as string stored somewhere. You will need custom strigify callback to differentiate which should be converted back to date on parse and which should be a string.

          Still, this is not pure json this way, it's your own transformations

    • fendrak 1961 days ago
      Indeed it can, but I believe the point is that it's not round-tripable, meaning you can't parse the resulting serialized JSON back into its original form, the one that contained the Date objects.

      If you could do so, JSON.stringify/parse would be a convenient way to do a deep clone.

      • simlevesque 1961 days ago
        Yeah that's right. All I'm saying is that no matter what, the following sentence is false:

        > An example of a non-serializable value type is the Date object - it is printed in a non ISO-standard format and cannot be parsed back to its original value :(.

      • zachrose 1961 days ago
        It would be cool if there was a webpage for “Is your function ______?” with a list of things that functions can be:

        - Roundtrip-able (reversible?)

        - Pure

        - Effectful

        - Mutation-doing

        - Idempotent/Nullipotent

        - Algebraically closed over the set of its inputs

        - Et c.

        Every time I hear about a new one I wish I had such a list

        • phiresky 1961 days ago
          > - Roundtrip-able (reversible?)

          You mean injective

        • kccqzy 1961 days ago
          > Roundtrip-able (reversible?)

          Involution, if you mean calling a function twice will give the original input.