Show HN: Minimal Fibers

(github.com)

53 points | by bgongfu 2140 days ago

2 comments

  • dfbrown 2140 days ago
    Worth noting that ucontext is quite slow (at least on linux): https://www.boost.org/doc/libs/1_67_0/libs/context/doc/html/...
    • newnewpdro 2140 days ago
      It's fast enough for many applications.

      I think ucontext is an excellent starting point for a general implementation. You just abstract it with a thin veneer and adopt faster implementations as needed where applicable.

    • bgongfu 2140 days ago
      It's slower than Boost Context for sure, but still around 10x faster than (ab)using Pthreads on Linux. Until someone releases a faster standalone C library it's still the fastest portable solution for projects like Cixl that can't afford dragging C++ around.
      • manwe150 2140 days ago
        There's already many fast, portable, standalone C libraries for this. For instance, see table 1 of https://www.gnu.org/software/pth/rse-pmt.ps. The assembly code for a context switch is pretty minimal (on most platforms, it's just setjmp and longjmp) if you don't try manage the signal state. I would be surprised if there was significant variance in performance (other than whether it chooses to switch the signal mask). Additionally, many language runtimes directly support it without any extra effort on your part. So if you choose to instead use one of those language, you get a fast portable solution without needing to do any extra work to pick a support library (for example, D-Lang LDC, PyPy, Go, Julia).
        • spc476 2140 days ago
          I implemented coroutines for C with assembly [1] (x86 32 and 64 bit). I took advantage of the calling convention to cut down on the amount of state to save (4 registers for x86 32b and six for x86 64b). Mixing this with signals is probably unwise [2]. So far I've tested the code on Linux and Mac OS-X and it works (although I might not use it for C++ either).

          [1] https://github.com/spc476/C-Coroutines

          [2] In my not-so humble opinion, using signals at all is not wise.

          • gpderetta 2140 days ago
            The shortest contest switch sequence I could come up on x86-64 is three instructions:

              xchg  %rsp, %rdx
              leaq  1f(%rip), %rax
              jmp   *%rsi
            1:

            It it expect the target stack ptr/ip pair to be in rdx/rsi and saves the current stack ptr and ip in rdx/rax. It does not save any register and uses gcc asm clobbers to instruct the compiler to save any other register.

            Code at [1]. The comments about hacks and ub is because I'm trying to transparently propagate exceptions across coroutines, otherwise the stack switching us fairly robust (although GCC specific).

            [1] https://github.com/gpderetta/delimited/blob/master/delimited...

          • girvo 2140 days ago
            Signals, done correctly, are hard. I agree with your NSHO for the most part. Neat code!
        • bgongfu 2140 days ago
          Not faster than dealing directly with ucontext from what I've seen; many wrap it directly and the rest tend to emulate using signals, setjmp and prayers.

          I would love to be wrong though...

          • manwe150 2140 days ago
            ucontext modifies the signal mask, requiring a syscall. That’s very expensive (as shown by the boost benchmark above), and usually unnecessary. “emulate” is a rather negative sounding way to describe running effective the same code as ucontext does (which also happens to typically be the same as sigsetjmp) - it’s not like ucontext has some privledged permissions. It’s “just” a context switch.
            • bgongfu 2140 days ago
              I was under the impression that Boost Context is more than that; at least that's what the amount of assembler code tells me; but I'll be the first to admit I don't have much patience for deciphering modern C++. I get that it's possible to invoke the same functionality as ucontext minus signal masks manually, but I'm not convinced it would save enough cycles to pay for the added complexity.
              • gpderetta 2140 days ago
                Saving the signal mask easily cost hundreds to thousand cycles. One or two order of magnitude more than the rest of contex switching.
            • bgongfu 2139 days ago
              For the record: I just benchmarked GnuPth with NULL sigmask against ucontext for the example in the post and it's slightly (3.4/3.2s) slower.
  • backpropaganda 2140 days ago
    Slightly OT, but do people know of any good small language which compiles to C? I'm not looking for big languages like C++, Nim, or Rust, but something which is essentially just C, but with some minimal modernization, such as perhaps being able to import modules, and doing away with header files.
    • bgongfu 2140 days ago
      Cixl? :)

      I realize the Forth & Lisp heritage might be a bit hard on the brain for those who haven't been exposed before, but what you're asking for is otherwise more or less what Cixl is aiming for.

      https://github.com/basic-gongfu/cixl#compiling

      • enriquto 2140 days ago
        Wow, cixl is a great thing! I love the Dijkstra quote against complexity that serves as inspiration for the language. However, you find this :

        > To build Cixl yourself, you'll need a reasonably modern GCC and CMake installed.

        I was very taken aback by this. Really? Why do you really need to depend on the cmake monstrosity? It's just a few lines of C without dependencies that they are distributing!

        If there is something about cixl that will scare people is the ridiculous dependence on cmake, not its clean syntax.

        • bgongfu 2140 days ago
          I hear you; I'm not overly excited with CMake either, or the C build system story in general.

          I spent several years trying to bend regular make into something I was happy with; then I spent several years on top of that using Rake, since at least it allowed me to say what I mean in a sane language. Compared to Rake, CMake is at least semi-standard, provides some kind of macro for most things I want to do, and mostly stays out of my way.

          But you definitely have a point when it comes to simplicity and project fit. If someone would be willing to step up and help translate the makefile into something that doesn't look horrible, I'd be more than happy to let it go. Otherwise we'll have to wait until I get enough round tuits, which could take a while given how much remains to be done in Cixl.

          • enriquto 2139 days ago
            Often it is disheartening to bend make to your bidding. But cixl has a neatly arranged tree and the makefile to build it is trivial. The following lines suffice (and by running 'make test -j' you compile everything in parallel and run all the tests) :

                CFLAGS   = -Isrc -O2
                LDLIBS   = -ldl -lm
            
                SRC      = $(shell ls src/cixl/*.c src/cixl/lib/*.c)
                OBJ      = $(SRC:%.c=%.o)
            
                src/main : src/main.o $(OBJ)
            
                clean    : ; $(RM) $(OBJ) src/main
            
                test: src/main ; for i in tests/*; do ./src/main<$$i; done
            
            
            would you accept such a patch in your project?
            • bgongfu 2139 days ago
              Not bad; my make-fu was never that spectacular to begin with and it hasn't been aging well, this helps me a lot.

              Sure thing, if you feel like giving the entire makefile the same treatment I'd be delighted to accept it.

              Thanks for taking the time!

        • exikyut 2140 days ago
          What's so bad about cmake? The first time I ever used it, I was very pleasantly impressed by the percentage system.

          I take it that cmake tried to tackle autotools, partially succeeded, and partly succumbed to autotools' all-devouring complexity? :)

          • enriquto 2140 days ago
            I just do not understand its point. Cmakefiles are much, much uglier than makefiles. And it takes several steps to compile a program, instead of a single one. And it seems larger than the operating system itself.

            I do not know what is "the percentage system", but I doubt it can be so wonderful to make me forget about all the other unnecessary problems.

            It may be justified to use cmake when it is really needed. But the cixl interpreter can be built by a five-line makefile.

            • exikyut 2140 days ago
              Ah, I see, a bit more tentativity has been filed away for when I finally check it out.

              I was regarding how it outputs progress indication, like

                [ 16%] Building C object ...
                [ 91%] Linking C executable ...
              
              etc (many lines elided). Back on the slow machine I was using when I was very new to Linux, progress indication ("how soon can I stop freaking out about this maybe not working") was very nice to have :D

              But (10 years on) I can understand it being icky to use. I guess, ugly as it is, it's a little more structured than automake is, and maybe that's its redeeming quality.

              Maybe build systems are the dead-centre of a "sour spot" (opposite of sweet spot) in computer science, with no truly nice solutions out there? (Genuinely curious. The exponential profusion of JS build systems suggests it's rather hard to solve.)

              The only explanation I have is that

              - Maybe the author is used to cmake

              - Maybe this project has Big Goals And Ventures™ syndrome - and, hence, all the enterprise build systems. (Hopefully not, it looks kinda nice)

              • enriquto 2140 days ago
                > Maybe build systems are the dead-centre of a "sour spot" (opposite of sweet spot) in computer science, with no truly nice solutions out there?

                I think it's the opposite. It is a very, very sweet spot for which many essentially perfect solutions exist. Then, it attracts all kinds of extreme bikeshedding and produces the monsters we see here and there.

                • exikyut 2140 days ago
                  Ooh, okay.

                  What do you think are the best/"essentially perfect" solution(s)?

                  (Said as someone who just figured out his first Makefile a couple days ago :P)

                  • enriquto 2139 days ago
                    make is essentially perfect for compiling small and moderately sized projects
      • backpropaganda 2140 days ago
        Yeah, the syntax is pretty out-of-the-world for me, but I'll give it a go. Thanks!
        • bgongfu 2139 days ago
          If you're serious about writing code; I strongly recommend learning some C, Forth & Common Lisp to get an idea of what's possible. Once you have those under your belt, appearances matter less and Cixl will look less crazy banana.
    • panic 2140 days ago
      Ion (https://github.com/pervognsen/bitwise/tree/master/ion) is a relatively new language with round-trippable conversion to C as a design goal.
      • exikyut 2140 days ago
        Ooooh, bitwise looks really interesting...
    • exikyut 2140 days ago
      Hmm. How would you do away with header files in a simple way if you wanted something similarly-scoped to C? Have the compiler vacuum in every source file at once to pick up all references, then discard via dead code elimination? That'd kill incremental compilation :/

      (I'm genuinely curious about the answer to this question - I'm also looking for something similar to C)

      • backpropaganda 2140 days ago
        Incremental compilation creates .o files for every compiled .c file, and so it could also create a .h file at the same time (perhaps fused in the .o file).
        • exikyut 2140 days ago
          Ooooh, duh. Takes notes
    • dom96 2140 days ago
      Nim is small enough to be used for this purpose. You can easily use it as a nice layer on top of C.

      Why do you consider it too big?

      • backpropaganda 2140 days ago
        I haven't looked at Nim in detail, but since Nim does garbage collection, it didn't read to me like a thin layer over C. For instance, is it possible to instantiate a struct on the stack? I'd also have liked the compiled C code to be readable, which if I'm not mistaken, Nim doesn't do.
        • dom96 2140 days ago
          > For instance, is it possible to instantiate a struct on the stack?

          Yeah, it is. You can easily avoid the GC as long as you don't mind managing memory yourself (unfortunately this means that you cannot use the stdlib either, or at least parts of it, if you disable the GC with the `--gc:none` flag you'll get warning messages from the compiler about procs that need the GC which is nice).

          > I'd also have liked the compiled C code to be readable, which if I'm not mistaken, Nim doesn't do.

          Yeah, that's one thing that Nim definitely doesn't do. I don't think there are many languages that do. It would be an interesting experiment to see how pretty you could make the generated C by either fixing things in the compiler or just running a C formatter on it.

          • backpropaganda 2140 days ago
            > unfortunately this means that you cannot use the stdlib either

            This is the kind of friction I expect from a language which uses GC, for my usecase. I think it might make sense for Nim to support a GC-less stdlib for people who do not want GC in their code.

            • theprotocol 2139 days ago
              This is giving me D flashbacks.
    • steveklabnik 2140 days ago
      I think you want Zig!
      • backpropaganda 2140 days ago
        Zig looks amazing! Thank you! While it doesn't compile to C, you read my mind that I actually wanted a C replacement rather than compile to C.
        • steveklabnik 2140 days ago
          Great! Yeah, that's what I was guessing, glad that I guessed correctly.
      • bgongfu 2140 days ago
        Isn't Zig based on LLVM rather than compiled to C?
        • steveklabnik 2140 days ago
          Yes, I didn’t assume that was the highest order bit here.
      • jitl 2140 days ago
        Zig explicitly does not depend on the C stdlib
        • steveklabnik 2140 days ago
          Yeah, this is true, I was responding mostly to the semantics aspect rather than that part.