Hey, I just saw this on the Github WASM thread, looks cool! I'm interested, but if I may request, you should highlight the missing/non-fully-functional keywords. That's my main concern, and from the main page it sounded like this was a fully working Go. Yet the Github issues seemed to highlight several missing keywords.
While this may not seem like a big issue, having to conceptually know what works and what doesn't is mental overhead, so reducing that overhead by making it super prominent would be great for me. Thanks!
This looks really cool btw, once I have a better idea on the completeness of this, I'll definitely try it with React! It's the one thing I failed with GopherJS on, React integration was difficult to keep performant. I'm excited :)
Thanks for the kind words. Yah I think you're right that I should make the current state more clear, not just where it's headed. FWIW, there's some more information on that lower down the page: https://mat.tm/joy/#next-steps
Hey, great work! Just a heads up about licensing. JavaScript code fragments that you use inside the compiler to produce the final output are technically part of the source code, and thus the output is also GPLv3 licensed as a derived work.
That's a known GPL quirk, and other GPL-licensed compilers like GCC use a separate variant of GPL to avoid that collision.
You can see how GNU Bison does it: https://git.savannah.gnu.org/cgit/bison.git/tree/src/parse-g... since they put fragments of Bison code to the final generate source file. That's exactly what Joy does to generate JavaScript, so you should put similar notice to the repository.
I command-F searched for GopherJS, looking for the obvious question -- how is this different? -- but found nothing besides crediting it as “inspiration”.
So, how is this different?
EDIT: asked in the Golang slack. Will update when I find out the answer.
> GopherJS emulates a 32-bit environment. This means that int, uint and uintptr have a precision of 32 bits. However, the explicit 64-bit integer types int64 and uint64 are supported. The GOARCH value of GopherJS is "js".
Joy doesn't seem to be:
> [Joy ships] a minimal runtime only when it's needed
> Most existing Go code does not yet compile to Javascript. This is because most of the standard library still needs to be translated to Javascript. This will be a focus of the 2.0 release. Signup for the mailing list to follow along with the progress.
This has some obvious consequences in how they differ. For example, if I look at [0][1], GopherJS results in a huge amount of code bloat because it includes a translation of the Go runtime. A basic example 33 LOC grows to 44 LOC on its own, but with the GopherJS environment that becomes:
> 1470 LOC and 45kb, uncompressed and unminified. The bulk of which is the builtin library.
It will only compile what it needs to. So if you declare types that are never used, they won't show up in the resulting code. This goes for core packages too. If I change that code so it requires "fmt", the result explodes to 12845 LOC and 624kb ("fmt" imports a LOT of stuff).
(In GopherJS' defense, minification slims it down to 21kb, and the example probably is missing a production code compiler flag)
The examples on the Joy webpage look a lot more minification-friendly and easier to integrate with other JavaScript.
GopherJS is _not_ an emulator. Saying that it emulates a 32-bit environment means that `int` (which is architecture-specific in Go) is 4 bytes (compared to 8-bytes on amd64), and means that int64 is supported and does what you expect.
As a counterexample -- it is not possible to have a 33 LOC example that uses goroutines to only expand to 44 LOC. GopherJS goes to extremes to make sure that your Go code works as expected in the browser.
My company has a few libraries (including a language parser/compiler) currently written in Go, and we have a need for the exact same functionality in the browser and in other places (e.g., Node.js apps). Rather than port everything to JS and have two codebases to maintain, we tried translating with GopherJS, which seemed promising at first. But the emitted code was enormous, and it pulled in parts of the standard library that couldn't be transpiled. (Our code had limited dependencies, but our main problem was the "fmt" package, which, if I remember correctly, is somewhat special and not something you can just omit.) In the end, it was too much work for a small company with our limited resources.
I'll be back on this thread tomorrow to answer any lingering questions. Feel free to open an issue on github with any additional questions or ping me at twitter.com/@mattmueller.
I came here to say the same thing in re: the name conflict with Joy PL. I'm currently working on a Joy interpreter. I'm just getting it ready to post to PyPI (it's implemented in Python.)
It's pretty obscure, but I have high hopes for the language. It has some amazing qualities.
I thought about it a little more and I don't think there should be an issue. This project is a compiler, not a language – Go is the language. It's analogous to Javascript and Babel.
Manfred taught me mathematical logic back in the mid 90s. It was only much later browsing functional programming topics that I discovered his language and that he had passed away.
My mate has an awesome story about providing technical support to him on campus. I hope neither he nor the deceased mind me relating it.
"Hello. Tech support? This is Manfred von Thun. My computer is not working."
"OK, no problem, do you have a PC or a Mac?"
"Neither."
<Pause>
"Neither?"
"Yes. Neither."
"Ok, what do you have, then?"
"I have a terminal which connects to the mainframe."
<Pause>
"OK, I'll be right over."
Sure enough, he gets there and there's some godforsaken yellowing text-only terminal thing with a wire plugged into some weird port he'd never seen before, presumably wending its way into the bowels of the university to some long-lost, dusty piece of big iron from computing's distant past.
My mate was incredulous. "So what do you do with this, exactly?"
"I write papers, I write programs, I make web pages."
"Web pages!?"
"Yes, web pages. But I don't believe in graphics."
In spite of his protestations about GUIs, he was eventually given a brand spanking new eMac (this was around about the time he was creating Joy). Once he saw Terminal.app, he no longer seemed particularly concerned.
Every staff member had one of those. They opened new computer labs at La Trobe in about 1995 but before that it was VTxxx terminals accessing a mixture of VMS or *nix, I believe.
I remember using vi in typesetting a paper with LaTeX before print previewing a DVI on one of the PC or Mac machines.
Awesome, do you have any more info about them you'd be willing to share? By the time my mate was around they weren't common enough in the humanities dept for him to have ever seen one before Manfred's. He's likely at least as curious as I am!
I was at La Trobe 1992-6 and Manfred taught me in 95 or 96.
I remember a mixture of VMS, Ultrix and I think SunOS. A monochrome screen with curses to do your email, browse gopher and text-only web, compile and run programs (in Manfred's case) and log in from any terminal on campus. There were a couple of monochrome Xterms in the computer centre.
But desktop computers had obviously taken over by your friend's era.
He was a European gentleman in a remote land with a quirky sense of humour and generous spirit; not your typical stuffy academic in tweed. :) And he had an awesome beard, long before hipsters made them fashionable!
The tribute pages below provide more insights from others.
I was studying in the mathematics department at the time, which was on the other side of the campus from the philosophy department. But there was considerable overlap between pure maths and formal logic. Godel's incompleteness theorem was one of the topics we proved and provided a good theoretical foundation for higher order logic.
Yes, Joy is amazing. I have been working through the Advent of Code puzzles with it and the code keeps coming out error-free. It delivers on Backus' idea of algebra of programs (aka "Functional Programming") in a direct and entertaining way. It also seems amenable to efficient compilation.
I'm pretty sure Joy is a kind of "silver bullet" for programming, in the sense that, as "software is eating the world", Joy will eat software.
Looks interesting but I think WebAssembly remains the more promising path for compiling to web browsers at this point. x to JS transpilers are likely on the way out.
From an issue in the GitHub, it sounds like GopherJS has a slightly different goal; it is attempting to make it possible to compile existing Go libraries / projects, whereas Joy is more focused on new projects built specifically to run in the browser.
There is a statement in that document that says that much of the standard library hasn’t been translated but isn’t most of the Go standard library written in Go? Why can’t Joy convert the standard library?
What a coincidence. I was thinking when would something like this be out yesterday and it stayed just a thought. Didn't bother to google and now I see it here. Something I would certainly love to try.
> Translate idiomatic Go into concise Javascript that works in every browser. Use Go's type system and world-class tooling to build large web applications with confidence.
Thanks, but for type system we already have type script. Secondly, the post powerful Golang feature which is you know co-routines can't be translated to javascript
I was a heavy Typescript user and a big fan. I personally found Go's type system and overall ecosystem to be much easier to grok, especially when it came to interacting with 3rd party packages. If you read through https://mat.tm/joy/#why you'll read more about my perspective there.
Actually to my surprise, goroutines and channels can be modeled quite well in Javascript! Right now the compiler uses async/await but for 1.0, I think it's possible to compile it down to ES3.
Let me attempt to clarify a few things with fresh eyes:
Javascript doesn't have threads, so true coroutines not really possible (w/o webworkers). If you lock up the event loop in one of Joy's goroutines, the other's won't proceed.
This limitation doesn't mean that we can't use the CSP concurrency model in an awesome way in Javascript. If you search NPM for csp or channel, you'll find some good examples.
For the Go programs I've come across using goroutines, calling an async function like this (async () => {})() is a good enough approximation for frontend development.
You may lose out on true parallelism and better error handling, but that's the JS environment we're targeting.
Hi Matt! First congratulations. This is a very interesting project, one that I look forward to trying out.
I'm wondering if Promises are the right way to compile goroutines though. Promises don't run in parallel, and they actually do block; they don't defer to the event loop. I wonder if compiling to web workers would be possible though...
Wait, async/await and co-rotines are completely different concepts, aren't they? the former just separate the moment of execution from the moment of receiving the result, the latter allow to execute the code in parallel. In browser the parallelism is only possible with web-worker.
Did you try Angular 4 + TypeScript? I recently switched to it after years in React and that's amazing, google really did it right this time. After using it for a while I don't feel any need in any additional tools.
The author stated below that co-routines are translated to async/await. for me it seems to be completely different concepts because golang co-rotines allow to execute the code in parallel which async/await can't do.
Given that the Go runtime will put any number of goroutines on any number of threads, it seems like being limited to one thread in a JS environment shouldn't break anything. The concurrency model works regardless of the number of underlying threads.
That's right, concurrency model does not break anything but still there are multiple concerns :
1) async/await are not supported in ES5, so after translating code from Go to JS I would also need to babelize it!
2) parallelism is not preserved
3) maybe it's just me, but in fat client apps it's all about event handling and data flow, co-rotines are not very useful there. But in Golang it's the most powerful feature ever(IMHO)
> in fat client apps it's all about event handling and data flow, co-rotines are not very useful there.
I would suggest that coroutines can solve event handling problems quite nicely. This is not exactly a novel idea either. There are toolkits in the wild that are designed to work exactly like that. From what I understand, there hasn't been any serious exploration of building UIs in Go, but I think its native coroutine/channel support has the potential to provide some interesting ways to handle events that are generally shied away from in languages where coroutines are not as easily used.
1) I don't think it actually uses async/await as it explicitly targets ES3. It's probably just using callbacks or something.
2) Parallelism isn't guaranteed by goroutines anyways, and this is targeted towards building new apps more than it is reusing existing Go code, so it shouldn't really be that much of an issue. Most web apps/SPAs have no need for parallelism.
1) It does not really matter that you think because the author said the following:
"Actually to my surprise, goroutines and channels can be modeled quite well in Javascript! Right now the compiler uses async/await but for 1.0, I think it's possible to compile it down to ES3."
I suspect "works" is strong - if there's no preemption, it's very easy to hit a deadlock. Consider a program that e.g. mines bitcoin on one goroutine and displays UI on the other. On any normal platform, the UI goroutine would get scheduled on a thread at least occasionally, but if you only have one thread to go around and no preemption, the mining task could just keep mining forever.
How are the two threads communicating? Via a channel? Why can't you schedule when a value is pushed onto a channel? Why do you need preemption in this situation?
The first task is writing to the second when it's done with a computation, but not bothering to check input from the second while it's in the middle of a computation.
Well, this works as long as you're using a separate goroutine and the Go runtime creates more than one thread, right? My claim is that a Go runtime that only creates one thread (which is effectively what Joy is) will not work the way you expect in this case.
Or, more specifically, that if you have N goroutines that consist of infinite loops of pure computation (or really, anything that doesn't involve receiving from a channel), and at least one other goroutine that's receiving from a channel, your runtime needs to generate at least N+1 OS threads or your last goroutine will never get scheduled.
That isn't any different from anything else in the browser though. It's not going to lock up anymore than async JS code would during CPU intensive operations. Which generally don't need to be done in browsers anyways.
I don't know too much about Go, but I think that background computations can be converted to web workers. Looks like Joy doesn't do it right now, but again, it doesn't seem fundamentally impossible.
(Also, it wouldn't surprise me if the majority of goroutines in practice were I/O heavy and not computationally heavy.)
While this may not seem like a big issue, having to conceptually know what works and what doesn't is mental overhead, so reducing that overhead by making it super prominent would be great for me. Thanks!
This looks really cool btw, once I have a better idea on the completeness of this, I'll definitely try it with React! It's the one thing I failed with GopherJS on, React integration was difficult to keep performant. I'm excited :)
I'll be here to answer any questions you might have!
That's a known GPL quirk, and other GPL-licensed compilers like GCC use a separate variant of GPL to avoid that collision.
You can see how GNU Bison does it: https://git.savannah.gnu.org/cgit/bison.git/tree/src/parse-g... since they put fragments of Bison code to the final generate source file. That's exactly what Joy does to generate JavaScript, so you should put similar notice to the repository.
Also, LICENCE file is not there.
So, how is this different?
EDIT: asked in the Golang slack. Will update when I find out the answer.
> GopherJS emulates a 32-bit environment. This means that int, uint and uintptr have a precision of 32 bits. However, the explicit 64-bit integer types int64 and uint64 are supported. The GOARCH value of GopherJS is "js".
Joy doesn't seem to be:
> [Joy ships] a minimal runtime only when it's needed
> Most existing Go code does not yet compile to Javascript. This is because most of the standard library still needs to be translated to Javascript. This will be a focus of the 2.0 release. Signup for the mailing list to follow along with the progress.
This has some obvious consequences in how they differ. For example, if I look at [0][1], GopherJS results in a huge amount of code bloat because it includes a translation of the Go runtime. A basic example 33 LOC grows to 44 LOC on its own, but with the GopherJS environment that becomes:
> 1470 LOC and 45kb, uncompressed and unminified. The bulk of which is the builtin library. It will only compile what it needs to. So if you declare types that are never used, they won't show up in the resulting code. This goes for core packages too. If I change that code so it requires "fmt", the result explodes to 12845 LOC and 624kb ("fmt" imports a LOT of stuff).
(In GopherJS' defense, minification slims it down to 21kb, and the example probably is missing a production code compiler flag)
The examples on the Joy webpage look a lot more minification-friendly and easier to integrate with other JavaScript.
[0] http://legacytotheedge.blogspot.se/2014/03/gopherjs-go-to-ja...
[1] https://gist.github.com/rolaveric/9407509
As a counterexample -- it is not possible to have a 33 LOC example that uses goroutines to only expand to 44 LOC. GopherJS goes to extremes to make sure that your Go code works as expected in the browser.
I am not very familiar with Go and am only barely passable at Javascript - but aren't these the wrong way round?
My company has a few libraries (including a language parser/compiler) currently written in Go, and we have a need for the exact same functionality in the browser and in other places (e.g., Node.js apps). Rather than port everything to JS and have two codebases to maintain, we tried translating with GopherJS, which seemed promising at first. But the emitted code was enormous, and it pulled in parts of the standard library that couldn't be transpiled. (Our code had limited dependencies, but our main problem was the "fmt" package, which, if I remember correctly, is somewhat special and not something you can just omit.) In the end, it was too much work for a small company with our limited resources.
Is Joy intended to serve this purpose?
I've created an issue for your purpose: https://github.com/matthewmueller/joy/issues/63
I definitely want to take the stdlib slow and not overdo it. If we can't make it work well in JS, we'll probably just leave it out.
I'll be back on this thread tomorrow to answer any lingering questions. Feel free to open an issue on github with any additional questions or ping me at twitter.com/@mattmueller.
Thanks everyone and good night!
https://en.wikipedia.org/wiki/Go!_(programming_language)
It's pretty obscure, but I have high hopes for the language. It has some amazing qualities.
I thought about it a little more and I don't think there should be an issue. This project is a compiler, not a language – Go is the language. It's analogous to Javascript and Babel.
Yeah, I don't actually think it will be a problem in practice. (And anyway I refer to my interpreter as "Joypy".)
It was sort of a knee-jerk reaction to seeing the name.
BTW, awesome idea and project! I hope it gets traction (at least until WASM "hits", eh?)
RIP.
"Hello. Tech support? This is Manfred von Thun. My computer is not working."
"OK, no problem, do you have a PC or a Mac?"
"Neither."
<Pause>
"Neither?"
"Yes. Neither."
"Ok, what do you have, then?"
"I have a terminal which connects to the mainframe."
<Pause>
"OK, I'll be right over."
Sure enough, he gets there and there's some godforsaken yellowing text-only terminal thing with a wire plugged into some weird port he'd never seen before, presumably wending its way into the bowels of the university to some long-lost, dusty piece of big iron from computing's distant past.
My mate was incredulous. "So what do you do with this, exactly?"
"I write papers, I write programs, I make web pages."
"Web pages!?"
"Yes, web pages. But I don't believe in graphics."
In spite of his protestations about GUIs, he was eventually given a brand spanking new eMac (this was around about the time he was creating Joy). Once he saw Terminal.app, he no longer seemed particularly concerned.
I remember using vi in typesetting a paper with LaTeX before print previewing a DVI on one of the PC or Mac machines.
I remember a mixture of VMS, Ultrix and I think SunOS. A monochrome screen with curses to do your email, browse gopher and text-only web, compile and run programs (in Manfred's case) and log in from any terminal on campus. There were a couple of monochrome Xterms in the computer centre. But desktop computers had obviously taken over by your friend's era.
The tribute pages below provide more insights from others.
I was studying in the mathematics department at the time, which was on the other side of the campus from the philosophy department. But there was considerable overlap between pure maths and formal logic. Godel's incompleteness theorem was one of the topics we proved and provided a good theoretical foundation for higher order logic.
http://hosted.verticalresponse.com/291390/14a69b5ba4/test/te... https://respectance.com/tribute/manfred-von-thun/
I'm pretty sure Joy is a kind of "silver bullet" for programming, in the sense that, as "software is eating the world", Joy will eat software.
I have some Jupyter Notebooks here (also in HTML and md formats; md is good for looking at the 'books in situ on github): https://github.com/calroc/joypy/tree/master/docs
I absolutely love the idea of transpiling other languages to readable (and therefore, debuggable and maintainable) Javascript.
Kudos and I'll be watching this project.
WASM is on the radar though: https://mat.tm/joy/#faq-webassembly
I almost used this software, but I'm going to have to go through the code before I do now.
I am looking forward to giving it a try, though.
Will fix it in the morning but I need to sleep now. Zzzz...
Thanks for the kind words and please try it again tomorrow :-)
I could be completely missing something, but mat.tm is down.
Thanks, but for type system we already have type script. Secondly, the post powerful Golang feature which is you know co-routines can't be translated to javascript
Actually to my surprise, goroutines and channels can be modeled quite well in Javascript! Right now the compiler uses async/await but for 1.0, I think it's possible to compile it down to ES3.
Javascript doesn't have threads, so true coroutines not really possible (w/o webworkers). If you lock up the event loop in one of Joy's goroutines, the other's won't proceed.
This limitation doesn't mean that we can't use the CSP concurrency model in an awesome way in Javascript. If you search NPM for csp or channel, you'll find some good examples.
For the Go programs I've come across using goroutines, calling an async function like this (async () => {})() is a good enough approximation for frontend development.
You may lose out on true parallelism and better error handling, but that's the JS environment we're targeting.
A coroutine is not a thread and an implementation of coroutines does not (necessarily) depend on threads. Goroutines can also run on a single thread.
I'm wondering if Promises are the right way to compile goroutines though. Promises don't run in parallel, and they actually do block; they don't defer to the event loop. I wonder if compiling to web workers would be possible though...
FYI GopherJS (the original Go -> JavaScript transpiler) has full support for goroutines. So "can't be translated to javascript" isn't quite right.
The post describes how the author has used, and liked, TypeScript but still wants Go's type system.
> Secondly, the post powerful Golang feature which is you know co-routines can't be translated to javascript
Why can't it be translated? A 5-second search of the git repo found this test case specifically checking whether goroutines are translated correctly:
https://github.com/matthewmueller/joy/blob/bb4ad647369331b55...
1) async/await are not supported in ES5, so after translating code from Go to JS I would also need to babelize it!
2) parallelism is not preserved
3) maybe it's just me, but in fat client apps it's all about event handling and data flow, co-rotines are not very useful there. But in Golang it's the most powerful feature ever(IMHO)
I would suggest that coroutines can solve event handling problems quite nicely. This is not exactly a novel idea either. There are toolkits in the wild that are designed to work exactly like that. From what I understand, there hasn't been any serious exploration of building UIs in Go, but I think its native coroutine/channel support has the potential to provide some interesting ways to handle events that are generally shied away from in languages where coroutines are not as easily used.
2) Parallelism isn't guaranteed by goroutines anyways, and this is targeted towards building new apps more than it is reusing existing Go code, so it shouldn't really be that much of an issue. Most web apps/SPAs have no need for parallelism.
"Actually to my surprise, goroutines and channels can be modeled quite well in Javascript! Right now the compiler uses async/await but for 1.0, I think it's possible to compile it down to ES3."
How are the two threads communicating? Via a channel? Why can't you schedule when a value is pushed onto a channel? Why do you need preemption in this situation?
(Maybe I'm wrong and Go handles this smoothly?)
Or, more specifically, that if you have N goroutines that consist of infinite loops of pure computation (or really, anything that doesn't involve receiving from a channel), and at least one other goroutine that's receiving from a channel, your runtime needs to generate at least N+1 OS threads or your last goroutine will never get scheduled.
https://github.com/golang/go/issues/11462 seems to be saying the same thing (by setting GOMAXPROCS to 1).
(Also, it wouldn't surprise me if the majority of goroutines in practice were I/O heavy and not computationally heavy.)
This is the key word. It is never guaranteed, so apps don't require it.