A proof that Unix utility sed is Turing complete

(catonmat.net)

202 points | by CrankyBear 1892 days ago

12 comments

  • samat 1891 days ago
    In a course on algorithms I took in Moscow State Uni (Russia), we were told there are two ways to describe a Turing machine:

    - One with a tape & state transition table - A string & a list of string replacements

    Then we proved this two definitions being equal.

    And programmed for each ‘system’ a little. I must admit, programming with ‘string replacements’ is much much more fun than doing ‘tape & table’.

    Is this just some Russian quirk or the same in the West CS courses, too?

    • schoen 1891 days ago
      I think the other formalism you're talking about is

      https://en.wikipedia.org/wiki/Markov_algorithm

      There are a number of different models of universal computation that were formulated in the 1930s or soon afterward and can all be proven to be equivalent in power. In CS courses in the U.S., people might learn about more than one of these and also prove that they're equivalent. But only the one with the tape and symbols is referred to as a "Turing machine" here; the other ones might be called "computation models", "computation formalisms", or something similar.

      I don't think that the string-rewriting model is as commonly taught over here, although I'm sure it's alluded to in discussions of rewriting in formal grammars

      https://en.wikipedia.org/wiki/Rewriting

      Interestingly, the Markov who devised this model of computation is apparently the son of the Markov who studied Markov chains and Markov processes.

      • throwawayRO12 1891 days ago
        You are right. In Romania we also prove that different computational models are equivalent (Turing machine, lambda calculus, Markov machine, logic programming)
        • abhishekjha 1891 days ago
          >prove that different computational models are equivalent (Turing machine, lambda calculus, Markov machine, logic programming)

          Is there a text which guides how to write such a proof?

          • theonemind 1890 days ago
            i remember covering this sort of thing in my automata theory class. we used this textbook: https://www.amazon.com/Automata-Computability-Complexity-The...

            I can't remember if it specifically shows such a proof in there, but you might look at courseware for automata theory and such.

            In general, from what I remember, you would use a general technique called "reduction", where you try to make two problems equivalent, "reducing" solving one to the problem of solving the other one, kind of mapping one problem on to another, so that solving one solves them both. So, you reduce running an arbitrary Turing machine, to say, lambda calculus, almost like you write a "compiler" from a Turing machine with lambda calculus as the target language. Then, you reduce computing in lambda calculus to a Turing machine. So, then you know that each can compute what the other computes, and they can only compute the same things. If you only did one half, one reduction, you would only know, say, that a Turing machine could do everything you could do in lambda calculus, but it would leave the possibility that the Turing machine could compute things lambda calculus can't.

          • naniwaduni 1891 days ago
            Generally, implement/simulate one in the other.
    • Cu3PO42 1891 days ago
      In my formal languages course in Germany we covered various computational models and showed the equivalences to Turing Machines. We never built complex TMs by hand, only describing how they would work.

      I did, however, build larger TMs by hand in a high school course.

      • gnulinux 1890 days ago
        Uh in a logic class, I had to build a really large TM that computes the Collatz Function (3n+1 or //2 depending on arity) and it was a very menial task. To debug my TM, I built a simple TM smulator in python and wrote unittests.
    • lqet 1891 days ago
      > Is this just some Russian quirk or the same in the West CS courses, too?

      I spent months building TMs (also a lot of DFAs) and equivalents as an undergraduate student (which was a lot of fun), and personally I always thought of a TM as a string character replacement machine. The set of alphabet symbols for a TM is of course not restricted to {0, 1}, you can also chose all UTF-8 characters if you like.

    • agumonkey 1891 days ago
      my brain always preferred rewriting systems to naive TM interpretation

      anybody else ?

    • porpoisely 1890 days ago
      We learn this in our Theory of Computation classes. Starting with basic deterministic finite automata to nondeterministic to regular languages to context free grammars to pushdown automatas to turing machines and halting problem.
  • wilsonrocks 1891 days ago
    Has anyone written a kind of Turing machine gateway API, where people write an adaptor to the API and you can then use the gateway to use any Turing machine backend to run any other(Turing complete) program.

    Would be interesting to see e.g. how fast Microsoft Word runs on a sed based backend.

    • daveFNbuck 1891 days ago
      You're basically describing compilers. Microsoft Word probably wouldn't run on sed because it requires things outside of the capabilities of a Turing machine, like being able to produce a GUI.
      • ascar 1891 days ago
        Considering a GUI runs on a turing machine, how is it outside the capabilities of one? Isn't everything we do on computers exactly within the capabilities of a turing machine and just about the ingenuity to create a program that does what we want and about its efficiency?

        In other words, if sed is turing complete, this means you could run any other turing complete program on it.

        Producing a GUI is also just telling your CPU (a sophisticated turing machine) to format data and how to send the formated data to a display.

        EDIT: what justifies the downvotes? Sure the display is not part of the turing machine, but the display is also not part of the GUI. The GUI is a software component that can be created by a turing machine. You wouldn't disagree with me for stating a GUI can be created through a C or Java program.

        • adrianN 1891 days ago
          Turing machines don't have displays or keyboards or network adapters. They have an infinite tape and compute (partial) functions from input words to output words, written on that tape. There are many things you can do that a TM can't do. Interactive computing is one of them.

          You can change the definition and add those devices, or you can simulate those devices on your tape, but simulation is not reality.

          • ascar 1891 days ago
            I never said something contrary to what you describe. The display or the mouse or keyboard is not part of the turing machine. But we communicate with these devices through a turing machine, e.g. the CPU. We interpret the signals or create visual output using a turing machine. A GUI is a software component and not some hardware. So if sed is turing complete, why shouldn't it be able to run Word, which is also running only on a turing machine. In other words: if sed is turing complete and C is turing complete, we could write a compiler from C to sed and thereby run any program written in C on sed (definitely with abysmal performance, but that's not the point).

            I'm not saying it's easy, efficient or even desirable to do so, but if sed is turing complete this means we could write a compiler to run any other turing complete program on it. That this process might need basically recompiling a whole operating system might be true, but it doesn't contradict the theory.

            I don't understand the downvotes here. I might very well be wrong, but your answer isn't giving an explanation for that.

            • adrianN 1891 days ago
              Oh I see, you want to add an interpretation level that takes the state tuple of the TM and translates it to pixels or your screen and patches the inputs like keyboard and mouse back into the TM. You can do that of course.
              • ascar 1891 days ago
                Not really the sate tuple. More like writing to certain positions of the tape results in physical I/O, e.g sending the bitstream over a cable to the display. This phyiscal I/O is outside of the capabilities of the turing machine, but it controls it through writing/reading from the tape.

                A turing machine is simplified a finite state automata with random access memory and you can use that memory to setup communication. Yes the communication isn't part of the turing machine anymore, but our computer programs dictate what to communicate and we know what to send to create a certain output. That's what I meant with ingenuity to create these programs.

                There is no real difference between any turing complete programming language or program apart from the libraries that help us do what we want.

                That Word uses a GUI and sed uses a terminal/console is no difference at all in that regard. Both are a graphical user interface. It's just that Word dictates exactly what it wants to display, while sed just sends text and lets the console dictate how to display that text. Both is using a turing machine to create the data that finally creates an image on our display.

            • saagarjha 1891 days ago
              CPUs are not simple Turing machines; they produce side effects as a result of their computation and interact with peripheral devices.
              • ascar 1891 days ago
                Turing machines are finite state automatas with random read and write access to a tape (i.e. memory).

                One turing machine can write to a certain position on a tape, while another turing machine can read from that position on the tape. At that point we communicated between two turing machines. Add some electric engineering to allow this tape position to send electrical signals from one place to another other a wire and you have peripheral devices. Yes actually displaying something through a display is thus more than a turing machine.

                But the topic at hand was between the different capabilities of one turing complete program/language (Word) and another turing complete program/language (sed) executing on the same turing machine. The definition of turing complete is that anything the first program can do, the second can do too.

                • saagarjha 1891 days ago
                  No, because the two Turing machines run in different contexts, one of which is unable to access a graphical display.
                  • ascar 1891 days ago
                    < one of which is unable to access a graphical display.

                    at that point we are only speaking about access control. It's very well possible to run sed in a context, so the bytes it outputs can be used to display information. In fact that's exactly what the console does. It interprets the output of sed as text and displays it on the screen. Nothing forces you to display the sed output in a console.

                    Digital displays only get bytes as input and output visible light for humans. These bytes are mostly an arbitrary format optimized for performance. Sed can output arbitrary bytes too based on an input program. That's what "sed is turing complete" tells us.

                    • saagarjha 1891 days ago
                      > at that point we are only speaking about access control

                      That’s the entire point. sed cannot run Microsoft Word because it cannot control the hardware in the right way. Sure, you could hook up sed to do this, but that would be kind of stupid and useful only in a purely academic sense.

                      • chrisweekly 1891 days ago
                        I'm with @ascar on this. The discussion started with an assertion (paraphrased) "Word couldn't run on sed because it requires things beyond a turing machine". This seems nonsensical to me; turing-complete, by definition, means it can represent any software program or set of programs, including of course an operating system, compiler, etc. To claim that MS Word somehow exists outside this context, because it happens to produce ones and zeros corresponding to a GUI rather than a CLI, is clearly incorrect. Of course it's "purely academic"! Nobody said it'd be smart or efficient to implement. But that's not the question under discussion! No fair shifting the argument / moving the goalposts like that.
                        • daveFNbuck 1890 days ago
                          My original point was that you couldn't write sed code that would result in a Word window opening up. This would certainly be possible if you wrote a sophisticated series of drivers and adapters around sed, but sed alone isn't going to get you there. Just being Turing-complete doesn't get you to the ability to run something like Word.

                          I'd also note that because physical computers have finite storage, you could by the same logic implement Word in a finite-state machine. You could implement the transitions of a finite-state machine using symbolic links in the filesystem. With enough of a surrounding framework, ls -l could be used to do transitions between machine states and drive the program.

                          So you can write Word in ls in the same way you can write it in sed.

                          • ascar 1890 days ago
                            > Just being Turing-complete doesn't get you to the ability to run something like Word.

                            In fact it does. That's the whole point of turing-completeness. It is actually possible to create output with sed that results in executable bytecode similar to C. You would need a sed to bytecode compiler. You could even write this compiler in sed (once you have something else that can compile sed). Why? Because sed is turing complete. Yes, it's very unpractical to do so, but you could do it even on current hardware.

                            > I'd also note that because physical computers have finite storage, you could by the same logic implement Word in a finite-state machine.

                            actually you can't. Word itself is turing complete, i.e. you can theoretically program with Word. So no you can't implement word with a finite-state machine, because it can't reflect the theoretically infinite states an arbitrary input program can create. You need a turing machine for that.

                            > So you can write Word in ls in the same way you can write it in sed.

                            No. That's missing the point of sed being turing complete.

                            • daveFNbuck 1890 days ago
                              > It is actually possible to create output with sed that results in executable bytecode similar to C.

                              The executable bytecode for Word is a constant. cat can output that.

                              > Word itself is turing complete

                              Word on a physical computer doesn't have access to an infinite tape. It can only reach a finite number of states, so you can simulate Word as run on a physical machine using a finite state machine with perfect fidelity.

                              • c22 1890 days ago
                                Nothing on a physical anything has access to an infinite tape. So nothing is a turing machine and the phrase is meaningless?
                                • daveFNbuck 1890 days ago
                                  It's a model. The phrase is meaningful as far as its features correspond well to features of the thing you're trying to model. Turing machines are pretty good models for which functions are computable on something like a model computer. They're not good models for interactive graphical applications.
                                • saagarjha 1890 days ago
                                  Technically, yes.
                          • ascar 1890 days ago
                            we reached the depth limit of comments on the other answer. This will be my last reply, but you can still reply to this:

                            >> It is actually possible to create output with sed that results in executable bytecode similar to C.

                            > The executable bytecode for Word is a constant. cat can output that.

                            I didn't mean output in the sense of printing it to a screen. What I wanted to say is that you can write anything in sed that you can write in C and that you could create a compiler for sed like you can create a C compiler, to run arbitary sed programs on current hardware.

                            >> Word itself is turing complete

                            > Word on a physical computer doesn't have access to an infinite tape. It can only reach a finite number of states, so you can simulate Word as run on a physical machine using a finite state machine with perfect fidelity.

                            No, I'm pretty sure you can't. A finite-state machines can not even parse context-free (Chomsky type-2) languages, so it definitely can not execute a general purpose program written in Word, even with a finite amount of memory. And as we can write general purpose programs in Word (and in sed, because they are both turing complete), you could never create a finite-state machine that can do everything that Word can do.

                            But yes, we usually ignore that our computers don't have infinite tape, as even the possible permutations for input programs of just 1kb are for our purposes basically endless (2^1024).

                            The point with C, C++, Java, Python, Word and sed being turing complete is that you can write general purpose programs for them to execute. You can not write general purpose programs for finite-state machines, you can not even create finite-state machines for every program. That's the interesting part of identifying if something is turing complete, even though it has no practical use it's not, because there isn't the necessary envrionment and it's never as efficient as the general purpose programming languages we built.

                            • daveFNbuck 1890 days ago
                              > What I wanted to say is that you can write anything in sed that you can write in C

                              This actually doesn't follow from sed being Turing-complete, and I don't think it's true. The construction to show that sed is Turing-complete requires transforming the input into one that sed can then perform computations on. I think a more clear example is Lambda calculus.

                              You can't write a Lambda calculus expression that increments a binary number. Lambda calculus can only operate on lambda expressions. Turing-completeness just means it's capable of computing the same sort of functions, not that it can match the input and output formats.

                              > you could create a compiler for sed like you can create a C compiler, to run arbitary sed programs on current hardware.

                              My point is that no input to sed will result in a Word window opening. sed can perform all the same calculations, but there's more to Word than that. I think we're agreed on this point. We're also agreed that you could embed sed in an appropriate framework and produce that window. I think our disagreement between these is mostly semantics, so I'm happy to stop arguing about it if you're done.

                              > No, I'm pretty sure you can't. A finite-state machines can not even parse context-free (Chomsky type-2) languages, so it definitely can not represent a general purpose program written in Word, even with a finite amount of memory.

                              Finite state machines can do anything that requires a finite amount of memory. For example, finite state machines can't recognize balanced parentheses. But they can recognize balanced parentheses in strings of length up to 10. There are only 2047 strings of parentheses of length up to 10, so you can have a state for each one and mark the valid ones as accepting states.

                              If we model the state of your computer, including every bit in ram, storage, caches, etc., then we know which state your computer will be in after the next clock cycle. We can therefore model your computer as a finite state machine, where each state represents the state of your computer at a given moment, and the transition is the result of your computer operating for a clock cycle.

                              There are a finite amount of possible states, and the transitions are deterministic. This defines a finite state machine.

                              You can't define a finite-state machine that can do everything that Word can do in principle, but you can define one that can do everything that Word can do on your computer, or in a VM. There would be way more states in this machine than there are atoms in the universe, but it's within the power of this theoretical construct.

                              • ascar 1890 days ago
                                > I think our disagreement between these is mostly semantics

                                Yes. I agree.

                                > Finite state machines can do anything that requires a finite amount of memory. For example, finite state machines can't recognize balanced parentheses. But they can recognize balanced parentheses in strings of length up to 10.

                                I already typed something similar before I changed my comment. I basically wanted to argue that it's theoretically correct, but we even dismiss it in computation theory, because modeling every state of a turing machine with just a few kilobyte of input is physically impossible.

                                But I was so unsure if that was actually true or there was a more fundamental problem that I convinced myself the "finite input" isn't the only thing that limits the capabilities of finite-state automatons vs pushdown automatons vs turing machines. I'm still not entirely sure, but I was thinking exactly the same as you explained.

                            • daveFNbuck 1890 days ago
                              I don't think HN has a depth limit. Sometimes I find the reply link doesn't show up under a comment, but I can usually see it if I refresh the page or click the permalink.
                              • ascar 1890 days ago
                                Yea, reply showed up later ¯\_(ツ)_/¯ i'll try the permalink next time
                        • foldr 1890 days ago
                          >turing-complete, by definition, means it can represent any software program or set of programs, including of course an operating system, compiler, etc.

                          Turing-complete just means that it can compute any function that a Turing machine can compute. Turing machines are universal computers, not universal machines.

                          My computer can compute F ⇒ a Turing machine can compute F (assuming the Church-Turing thesis).

                          My computer can do X ⇏ a Turing machine can do X.

                      • ascar 1890 days ago
                        Of course it's purely academic. The whole point of proving sed is turing complete is purely academic. It's interesting to know that other tools, which were designed for a very specific task in a very specific context, can be used as general purpose programming languages. There is no point discussing this in a practical context.
                        • c22 1890 days ago
                          It's not entirely academic, though, because surprise turing machines often lead to weird bugs or failures in assumptions about security.
                • JdeBP 1891 days ago
                  "automata" is already plural, note.
              • basementcat 1891 days ago
                Practically realizable CPU’s are just deterministic finite state automata with a very large number of states.
                • saagarjha 1891 days ago
                  Sure, but their states aren't quite what you'd consider to be states for an abstract Turing machine. There's a CPU state that also happens to set some pixels on a display to a certain value, for example.
                  • baddox 1891 days ago
                    So a display is simply a device you plug into some pins on the computer to format the computer’s state in a more human-friendly way. You could do the exact same thing with a Turing machine.
                    • saagarjha 1891 days ago
                      Yeah, but you don't with sed. Hence why it can't be used to run Microsoft Word.
                      • c22 1890 days ago
                        Why not? While sed is chuggng away at assembling your instructions you will have plenty of time to work on the problem. Just bulid a device that plugs into the serial port of your computer, run your sed-word amalgamation in a shell on the serial tty. Configure your device to interpret the ascii sequences from the serial terminal as pixels and output them as physical dots of light. Now for bonus points you can redesign the communication protocol to be more compact.
          • yason 1891 days ago
            Turing machines don't have displays or keyboards or network adapters.

            Nor do processors these days. To the cpu, keyboards, networks, and displays are just electronic signals and the cpu drives a logic to interpret and control those signals. The logic could as well be driven by a program running on a Turing machine, albeit awkwardly of course.

            You could also argue that a Unix tool that reads stdin and writes to stdout doesn't have displays or keyboards, and is limited in what it can do. Yet the input/output streams are an abstraction: if we hook the output stream to drive a display and connect the input stream to keyboard event generator the program can certainly drive a graphical interface even if particular Unix tool itself has absolutely no notion of a keyboard nor a display. It just sees bytes in, bytes out, as programmed.

            They have an infinite tape and compute (partial) functions from input words to output words, written on that tape. There are many things you can do that a TM can't do. Interactive computing is one of them.

            Turing machine is a theoretical model of a computing machine as far it goes to reality itself, but the model does involve the bare essentials to do meaningful work such as input and output. And it is that input and output are what would allow it to be hooked into something that is real.

            Given the realism of that we don't actually have infinite tapes either, any hypothetical application of a Turing machine would obviously manipulate the tape to give the program initial input and interpret the tape to read output in order to have the program do useful work.

            The notation of an infinite tape is just Turing machine's only way to handle input and output, but it is really an abstraction for any input and output.

            The Turing machine doesn't care where the data on the tape came from or if the output is interpreted as letters, pixels, or audio.

          • basementcat 1891 days ago
            You can define certain tape positions to be i/o ports.
          • astrobe_ 1890 days ago
            I think for the display a Turing machine can simulate a kind of video RAM on a specific location of the tape ; but I guess a single pixel would need a significant amount of cells.

            For inputs one could introduce memory-mapped peripheral registers, like the M68K does.

            The original definition of TM can be preserved if you consider that you are introducing the notion of an outside world that would look at the tape at the end of the TM run (for display etc.) and feed another tape with different initial cell states (peripheral register values) for the next run.

            > simulation is not reality.

            Virtual machines and emulators are however quite useful ; and it seems that for some people, simulating human intelligence is the Graal.

          • csomar 1890 days ago
            Full simulation is reality. A reality that you can not see (through your screen) but a reality indeed; on its own box.
        • jhanschoo 1890 days ago
          Note that Turing completeness says nothing about efficiency. It is possible that some models experience exponential slowdown or worse.
      • adriveatrain 1891 days ago
        Word is a set of COM objects, you can run it server side without the UI.
    • User23 1891 days ago
      This that you describe is essentially a Universal Turing Machine.
      • Quekid5 1890 days ago
        There's been a very weird discussion going on here based on the misconception that TMs are somehow equivalent to Interactive TMs[1]. In the usual theory of TMs there's no idea that there's "input" (e.g. another TM altering the tape) or anything similar. You just give it a starting position and the initial state of the tape and let it go.

        [1] https://en.wikipedia.org/wiki/Interactive_computation (or course I use the term Interactive TMs a bit loosely, but y'know...)

      • afiori 1891 days ago
        Yes, but it should be noted that a sed-interpreted word would also need to emulate the OS and the hardware, as I imagine that the syscall you can make from sed are limited
  • kccqzy 1891 days ago
    sed really is a powerful utility. I once wrote a tool in sed that strips HTML tags leaving just plain text as a fun exercise. Naturally it can't handle complicated cases but for many simple use cases it works. The code, though, is basically unreadable.
    • saagarjha 1891 days ago
      Given that sed is Turing complete, I'm sure if you tried harder you would have come up with a general solution ;)
      • mannykannot 1890 days ago
        Absolutely. One approach would be to take this TM implementation and program it to parse HTML.
    • airstrike 1891 days ago
      Any chance you could share that? Wouldn't mind giving it a whirl in this one side project I just started...
    • mirimir 1890 days ago
      Isn't it more or less a cliche that using sed etc for stripping HTML tags is unreliable and potentially hazardous?

      I've tried it. I could get it to apparently "work". But then I'd get some input that hosed it. Now I just use "w3m -dump". I mean, it's a browser.

      • noxToken 1890 days ago
        There's the classic Stack Overflow answer[0] about matching HTML tags. If you have a small subset of HTML, it can work out pretty well.

        [0]: https://stackoverflow.com/questions/1732348/regex-match-open...

        • mirimir 1890 days ago
          Thanks. That's what I was thinking of. Classic "this parrot is dead" riff.
      • mannykannot 1890 days ago
        The unreliability follows from it being "basically unreadable", which leads to it probably not doing what you intend. Whether it is hazardous depends on what you use it for.
    • james_s_tayler 1891 days ago
      I used to write web scrapers with curl/grep/sed back in the day. Fun times.
    • somacert 1890 days ago
      sed -e 's/<[^>]*>//g'
  • fouc 1891 days ago
    This article was submitted before in 2012. It was recently updated.

    from 7 years ago: https://news.ycombinator.com/item?id=3855616

  • nickdrozd 1891 days ago
    Here's a simple infinite loop in sed:

      echo loop | sed -e ':x' -e 'p ; bx'
  • qwerty456127 1891 days ago
    I really wish there were more Delphi/VB/WinForms-like visual RAD tools for Linux (Lazarus is just a way too primitive, I've tried it and it feels like neither the editor nor the language has undergone any improvement since 90es).
    • jim_bailie 1890 days ago
      And I love Lazarus/FreePascal for those very reasons. A simple and very performant tool set that I use quite extensively for a set of mission critical quantitative financial analysis applications.
  • gonzo41 1891 days ago
    It's not turing complete until someone writes a ray tracer in it!
  • Grue3 1890 days ago
    And that's a bad thing. It means not only that some sed programs never complete, but it's not even possible to tell whether any given sed program hangs or not. Not something you want from a text substitution utility.
    • dahart 1890 days ago
      Out of curiosity, have you ever been concerned to have a guarantee whether any program you’ve written or run would finish, regardless of run time?

      The halting problem is an abstract theoretical result that I’ve personally never been concerned about, in any language at all. Not even once in practice in 30 years of coding have I worried that I didn’t know whether a correct bug-free program would finish. I worry that programs take too long all the time, but that’s completely separate from the halting problem.

      If sed is a “bad thing”, just understand that string replacement is a “bad thing” and computers are a “bad thing”. There isn’t an alternative to doing any real computation without having Turing completeness.

      • Grue3 1889 days ago
        Yes, I have personally designed several guaranteed-to-halt DSLs for the exact reason that allowing Turing completeness where it's not needed is a bad idea. If you have a scripting language in another software, you don't want to allow the scripts to go into indefinite loops, that's just bad UI. You can't guarantee the user will write bug free code that halts.

        One of my non-Turing complete languages is Animstack [1]. It's a script within GIMP image editing software. Imagine accidentally creating an infinite loop while you're working on an image and having your image editor freeze forever. Well, you can't create an infinite loop in Animstack. It's guaranteed to finish in linear time (of the number of layers). That's intentional and I was careful to preserve this guarantee.

        [1] http://tshatrov.github.io/animstack.html

        • dahart 1889 days ago
          Your project looks really cool, but I’m unconvinced by your take on Turing completeness. It’s a mistake to conflate UI with computability, and a mistake to confuse computability with the ability to make programming mistakes or to write bugs.

          The academic idea of the halting problem is not to make good UI or save people from themselves or from bugs, it’s just a theoretical math problem, and it doesn’t mean anything useful in practice.

          You can write an infinite loop in C, and still hit Ctrl-C to kill it. You can screw up JavaScript and still close the tab. Turing completeness is not a problem in practice. If GIMP’s scripting or plugin environment doesn’t support killing a runaway script, then the bad UI is GIMP, not Turing completeness.

          Plus, lack of Turing completeness is no guarantee that a script will be fast and that it won’t be bad UI. The halting guarantee is not a user guarantee on responsiveness or on run time at all.

          Lack of Turing completeness might, on the other hand, severely limit real functionality, it might keep users of your language from being able to do cool things. You’re taking at least some kinds of higher level function composition off the table.

          Anyway, that said, I will try to check out your project, I’m a big fan of image generation DSLs (having written a couple myself for CG film production) as well as compositing and animation projects.

          • Grue3 1889 days ago
            I'm not convinced that Turing completeness is needed for real functionality. Can you name a "practical" function that is computable but not primitive recursive? Just because you can theoretically compute Ackermann function doesn't mean it has anything but academic interest.
            • dahart 1888 days ago
              Are you kidding? All you need for TC is conditional repetition, you don't need recursion. Loops & variables are the two things that make a language Turing complete. Almost everything interesting in the computing world is done using conditional loops.

              To use your Animstack project as an example, you're not allowing your users to write things like Perlin turbulence (multiple octaves of noise) without a loop, or custom convolutions, or a Mandelbrot fractal, or 2d fluid flow. There are all kinds of practical and fun things you're artificially restricting yourself from. Some of those things can be technically done without TC, but none typically are because it's inconvenient, and because TC doesn't cause any practical problems.

              Other examples that use Turing completeness in practice: UI event loops (all videogames, all of robotics, all servers of any kind), string replacement, sorting a list of arbitrary size, searching a list of arbitrary size, path tracing renderers, file i/o, regression and any non-linear optimization, I could go on for a very long time, the list is really long.

              I'd like to know what interesting things you think you can do without conditional loops in a convenient way. Not what's theoretically possible if you bend over backwards, but what's practical and easy. I feel like the list is pretty short.

              You also didn't answer the question. What is a real problem with TC? Killing runaway programs & infinite loops is not a problem in practice, and therefore neither is preventing people from writing infinite loops. So what's a problem with TC that I should care about? You can never guarantee people won't make mistakes, unless you make all their decisions for them. That's what non-TC languages normally feel like; a recipe book for pre-decided outcomes, not very powerful, not very interesting.

              Nor did you address the fact that a decidable non-TC language doesn't guarantee halting in any reasonable amount of time. So how is it actually helping you to prevent TC? It doesn't seem like it is.

              Here's a question- how many conditional loops did you use in the implementation of Animstack, under the hood?

              • Grue3 1888 days ago
                >Other examples that use Turing completeness in practice: UI event loops (all videogames, all of robotics, all servers of any kind), string replacement, sorting a list of arbitrary size, searching a list of arbitrary size, path tracing renderers, file i/o, regression and any non-linear optimization, I could go on for a very long time, the list is really long.

                A lot of these don't require Turing completeness. Anything that can be given runtime guarantee that's O(n^k) or even O(e^n) can be implemented without Turing completeness. Now if you need O(Ackerman-function) runtime, you gotta bring out the Turing machine, but I don't think such algorithms exist in practice.

                Consider that pure SQL is not Turing complete. You can "search lists of arbitrary size" in it. And you can construct pretty complex queries in it to do so. It's the most popular programming language around and most people don't use the parts that make it Turing complete.

                >Almost everything interesting in the computing world is done using conditional loops.

                There's almost nothing that requires conditional loops with arbitrary conditions. Loops in Python all have the form "for element in sequence" and you can go pretty far without using custom iterators that allow for infinite sequences, which is considered an advanced topic. A lot of people program in non-Turing complete subset of Python!

                > how many conditional loops did you use in the implementation of Animstack, under the hood?

                I actually can answer this easily. It's implemented in TinyScheme (the scripting language of GIMP) which doesn't have loops, only recursion. So technically zero. Scheme is Turing-complete of course, but I don't think it has to be to implement Animstack.

                • dahart 1888 days ago
                  Recursion is looping, so technically not zero, right?

                  There’s a lot of ignoring practice in your response. Python, C, O(n^k) algorithms, all of it is implemented with Turing Complete languages, whether it’s technically required or not. I don’t know the non-TC subset of Python you’re talking about, nor how many people are using it. Are they using it specifically to limit themselves to less powerful constructs, or for other reasons?

                  The web says SQL is in fact TC contrary to your claim. https://stackoverflow.com/questions/900055/is-sql-or-even-ts...

                  > There's almost nothing that requires conditional loops with arbitrary conditions.

                  The entire stack of tech you are using to type that sentence and send it over to Hacker news runs on intentionally infinite loops. The input element, the js in your browser, the event loop in your browser, your NIC and WiFi box, all the routers between you and HN, the web server responding to news.ycombinator.com. GIMP has an infinite loop waiting for user commands to launch your Animstack scripts.

                  My professional field is graphics, which is why I mentioned path tracing, and path tracing requires loops with arbitrary conditions, as do many statistical simulations. How would you write a non-TC memory manager? I don’t really know what you have in mind when you say almost nothing requires conditional loops, but I don’t see a lot of evidence of that being true. All video games run on while(1) loops.

                  You still didn’t answer my questions, so maybe this is as far as we need to go. Why does TC matter at all when you can kill a script? What is preventing TC buying your language when the programs are bug free? What is non-TC buying you when non-TC programs can take aribtrary amounts of time? Why is restricting the power of a language desirable? I havent seen a single reason yet that I buy.

                  • Grue3 1888 days ago
                    >The web says SQL is in fact TC contrary to your claim.

                    Did you even read the link? You need extensions to make SQL Turing complete. Your replies indicate to me you lack the necessary theoretical background to discuss Turing completeness, since pure SQL not being TC is such a basic fact you don't need to look it up on the web.

                    The fact that the entire tech stack is TC is not a proof that TC is needed for individual parts of the tech stack. Indeed, some of the components of the web stack, namely CSS, are not TC (no, please don't look it up on the web and come back with "CSS is turing complete" bullshit, they're clearly wrong because halting problem of CSS is decidable).

                    By the way Turing machine doesn't involve human interaction, so all your examples that involve interactivity (UI loops) are outside of scope of this discussion.

                    >What is preventing TC buying your language when the programs are bug free?

                    There's no such thing as bug free program. When you need to execute arbitrary programs, not being TC buys you a lot of guarantees that can be useful.

                    • dahart 1887 days ago
                      > When you need to execute arbitrary programs, not being TC buys you a lot of guarantees that can be useful.

                      Please, elaborate! What are these useful guarantees? Can you name some?

                      > Did you even read the link?

                      I think so... the first sentence says you don’t need an extension. The linked Mandelbrot set example is “SQL:2008 conformant” and depends on “nothing”. https://wiki.postgresql.org/wiki/Mandelbrot_set Is that using an extension?

                      > By the way Turing machine doesn’t involve human interaction, so all your examples that involve interactivity (UI loops) are outside of scope of this discussion.

                      Okay, so Turing completeness doesn’t apply to GIMP or Animstack at all, right? That’s what I was trying to suggest before. ;) Your own stated reason for removing TC from Animstack was UI.

                      Who says human input doesn’t apply? Alan Turing didn’t say that. Wikipedia says that interaction does apply: https://en.m.wikipedia.org/wiki/Turing_machine#Interaction “In principle, it is possible to model this [interaction] by having an external agent read from the tape and write to it at the same time as a Turing machine...”

                      > The fact that the entire stack is TC is not a proof that TC is needed

                      No, it’s just proof that TC languages are common and useful, and that not many practitioners would agree that TC is a “bad thing”.

                      > Indeed, some of the components of the web stack, namely CSS, are not TC

                      So? What does it prove if a few parts are non TC, when so many of them are?

                      CSS is for styling, not for writing programs, unlike Animstack and sed and the other examples you’ve been using. It tends to undermine your points a little bit to hold up CSS as an example of a non-TC language. I wouldn’t want to use CSS to compute things, so I don’t care if it’s Turing complete or not. For that matter, I don’t care whether any language is Turing complete, as long as I can get my work done easily, why should I?

                      > no, please don’t look it up on the web

                      Okay, I won’t. Just kidding, here’s the answer, not just a claim, but an implementation of Turing Completeness. So what’s clearly wrong about it? https://stackoverflow.com/a/5239256

                      To be honest, I don’t care what’s wrong with it. Or with sed or any other language. Not in purist academic theory. I wanted to hear if there are practical reasons to care, but since you won’t answer and I haven’t heard any, then I don’t care.

    • ben0x539 1890 days ago
      Never in my life have I been more concerned about whether a simple text substitution is going to halt than whether I can write it down reasonably concisely without having to resort to a more powerful scripting/programming language.
    • eponeponepon 1890 days ago
      If you just want a text substitution utility, then this is proof that you can implement that by using sed in a controlled manner. Why bemoan something for doing more things than you need it to right now? This is what general-purpose computing is all about.
      • Toenex 1890 days ago
        I do like the thought experiment that I might not want Turing completeness from a language in a specific use case.
    • a1369209993 1890 days ago
      Frankly, I really don't care about the supposed difference between:

        while(1) hang
        # and
        for(i < 9^9999) hang
    • jhanschoo 1890 days ago
      No, it's not possible to tell if arbitrary programs hang. But it is possible to tell if a program is of a restricted form that is known to terminate. This is how we know the many standard algorithms we frequently use terminate.
  • gammateam 1891 days ago
    Why is Turing completeness important to anyone?

    Asking for a friend

    • erfgh 1891 days ago
      It is if you can make a paper out of it and increase your publication count.
      • maxiepoo 1890 days ago
        I highly doubt that you can get a paper at any respectable conference or journal showing something is Turing Complete.
    • dwrodri 1890 days ago
      From the security point of view, it means that it can be used execute code from user space. Just about anything that's Turing complete should be on someone's list of potential attack vectors.
      • gammateam 1890 days ago
        Yes, I agree

        Why is it glorified? Unless all these researchers are really just intentionally giving code word to hackers for a target to exploit or weaponize, but it doesn't really seem like thats the point with these people.

    • slededit 1890 days ago
      It means the halting problem applies, and analysis of program behavior becomes much more difficult. Could be an interesting attack vector.
    • cyborgx7 1891 days ago
      Here is the (overly) simple explanation: Something being Turing complete means that it can compute anything that can be computed.
      • gammateam 1890 days ago
        I asked why it was important to anyone, not what it was

        why is this glorified

    • Zarath 1891 days ago
      Ask all the c++ metaprogramming people :)
  • otabdeveloper1 1891 days ago
    Basically, anything that can recurse is Turing-complete.
    • klmr 1891 days ago
      No. Only anything that can perform µ-recursion is Turing complete. Primitive recursion [1] is not enough. In practice virtually all languages that allow recursion allow its Turing complete form but it’s important to realise that other forms exist.

      [1] https://en.wikipedia.org/wiki/Primitive_recursive_function

      • deytempo 1891 days ago
        It fascinates me how few people can or are willing to explain concepts of computer science without using formal mathematical notational explanations. When asked, often they will say “that’s the only/best way to explain it” then I go and spend a few hours translating said concept from the formal math notational mess, finding that it’s indeed quite possible to explain it simply and even elegantly in either natural language or just a pseudo code example.
      • norswap 1890 days ago
        Could you give an example of primitive recursion that is not µ-recursion?
        • OskarS 1890 days ago
          Every primitive recursive function is also µ-recursive: the primitive recursive functions are a subset of µ-recursive functions. But the converse is not true: there are µ-recursive functions that are not primitive recursive. The canonical example is the Ackermann function. It can be shown that it grows faster than all primitive recursive functions, and is thus in itself not primitive recursive.

          Since the Ackermann function is obviously computable, and easily computable by a Turing machine, this implies that the primitive recursive functions are not Turing complete, and thus more limited than the µ-recursive functions.

        • zeroonetwothree 1890 days ago
          Primitive recursion is a subset of µ-recursion so there is no such example. I assume you just want an example of primitive recursion.

          The term is confusing if you are used to “recursion” in the context of programming. Primitive recursion basically corresponds to programs that don’t use recursion or unbounded loops. For example “compute the factorial of 55” or “sort this input list of at most 10000 integers”.

  • clanrebornsx 1891 days ago
    Isn't this the same racist guy who once made a WhatsApp kiosk and was surprised why it's getting popular among the people who really need it.
    • voltagex_ 1891 days ago
      I vouched for this because even though it's somewhat misguided, I want to know how this person came to this conclusion.

      This is the event referenced: https://scroll.in/magazine/864603/desperate-for-whatsapp-ten...

      He was unintentionally DDoSed. He made it right. What's the problem?

      • fouc 1891 days ago
        Interesting article!

        He definitely came across as incredibly ignorant about India in the beginning, and started out with a negative viewpoint.

        Ultimately he finally figured out what was going on and corrected his views, and was finally able to help indians use whatsapp from their JIO phones.

        He was almost too honest in the article, since it kind of makes him look bad. But I appreciate that honesty.

        • justwalt 1891 days ago
          I don’t think he was negative at all. I think his response was one that most everyone would have had.

          What do you think he could have done better?

    • mises 1891 days ago
      How was what he did racist?
    • solarkraft 1891 days ago
      Oh, the Cross-Browser-Testing-Tool-Abused-As-WhatsApp-Proxy guy! Good catch, nice!