I thought this was a potentially strong argument delivered weakly.
First, he poisons the well by talking about "integrity" and "intellectual honesty". This is a bad move at the beginning of a speech because it is far too easy to take it such that people with different conclusions are lacking in those qualities. Given his delivery, I suspect that he might actually believe that. It would be better if he just owned that this was his opinion and delivered it respectfully.
Second, the section on academic priorities was weak. He spent a large portion of the talk highlighting the mission statements of various languages, which is marketing language. But if Scala made no other changes other than its marketing language, it would not solve the problems he was highlighting. I thought too much time was spent on that part of the talk. There's another moment later in his rushed conclusion where he made stronger points that could have been fleshed out - how the academic influence leads to theoretical features that later get abandoned when the academic project is completed. (And I quite liked his revised marketing statement for Scala.)
I also didn't like that he brought up Cats vs scalaz as a flaw with the community, given that it became clear later in the talk that he had a clear bias on which was preferable. That seemed needlessly provocative.
I think the final conclusion of abandoning the OO path was interesting and fun to think about! I don't know if there's already an opinionated compiler plugin that discourages the more OO techniques while encouraging FP techniques, but I can see how that would be a lot of fun to play with, and educational in the same way that static analysis tools are. (I don't understand why "partial functions" was crossed out as something to abandon, though.)
Discouraging OO would be the worst thing Scala could do. Its one really unique capability is the fusion of OO and functional: it's a language that can go toe-to-toe with Haskell, yet also one that lets you switch from Java in a weekend. Take away the OO support and you'd have just another functional also-ran.
Sadly while the language itself has excellent OO support and Java interop, the community support just isn't there (by which I mainly mean: too many people like DeGoes will shout you down rather than trying to help you).
I dunno, I think the DeGoes has a point. Like he says near the end, there's this fiction that an OOP+FP hybrid will be loved by both OOP and FP programmers, when many times it actually means both will dislike it and the language will be mediocre at both goals.
My opinion -- and I mostly write Scala in my day job! -- is that Scala is in a truly difficult position. Its OOP (and Java) DNA hurts its FP aspects. If it goes full hardcore FP, it will probably lose relevance (because most programmers can't be bothered with it, and those who can will probably hope for Haskell on the JVM). If it stays like it is and tries to be a "better Java", it will keep losing ground to more practical and "easier" languages on the JVM, possibly including losing to Java itself!
PS: DeGoes didn't sound disrespectful to me, and bear in mind I don't know who he is. He seemed to be arguing out of practicality, not shouting people down. I don't if he is angrier in other forums, but his tone in this presentation sounded closer to a wake up call to me.
> If it stays like it is and tries to be a "better Java", it will keep losing ground to more practical and "easier" languages on the JVM, possibly including losing to Java itself!
Well, to my mind the FP is a means to practical ends. I'm not attached to Scala qua Scala, I'm attached to HKT and typeclasses because there are practical problems I can't solve without them. As long as Scala has those things and Java or Kotlin don't, Scala will have an advantage - and if Java does adopt those things I'd say mission accomplished. (I mean, Odersky's original goal was to improve Java, right?)
> there's this fiction that an OOP+FP hybrid will be loved by both OOP and FP programmers
That seems more of a strawman than a fiction: I've never seen anyone, especially in the Scala community, suggest this.
I've seen lots of people suggest that OOP+FP fusion is what allows Scala to provide an easy gradual on-ramp to FP for people and projects transitioning from Java, which is Scala’s key value proposition.
> I've seen lots of people suggest that OOP+FP fusion is what allows Scala to provide an easy gradual on-ramp to FP for people and projects transitioning from Java, which is Scala’s key value proposition.
I feel like I'm at the tail end of that on-ramp, and while I'm still as a whole really appreciating scala, I'm starting to wonder what the end-game is.
Now that I understand and am comfortable with all the functional stuff, I'm left wondering why I still have the OOP stuff, and why some of the FP stuff isn't more natural. I guess you still need some OOP for java interop, but the pure-scala ecosystem is pretty significant already.
I guess what I'm saying is that the easy transition from Java is only a benefit for bringing people in, not for a language as a mature ecosystem. And I'm thinking maybe I should get into one of the full-FP languages.
In fact, if anything, what's keeping me from making the switch is not any OOP features, but actually just imperative! Especially for performance-sensitive sections, and some rare-but-necessary data structures, it seems you really do want a small amount of mutable state.
> scala, I'm starting to wonder what the end-game is
The end game back in the early days of Scala was probably to provide a gradual on-ramp to FP for people and projects transitioning from Java, but it probably changed to a commercial purpose once more people got involved in its development. It's long since become a self-preserving product, and probably even tries to hinder people moving from FP-only Scala on to Haskell and the like.
I only spent a few months on Scala way back so don't know for sure, but I did see Apache Groovy go through a similar business re-purposing. It began as a JVM-based scripting language complementing Java, useful for glue code and testing, but later became a competitor to Java and other statically-compiled JVM languages, and even tried unsuccessfully to target Android.
Funny I was thinking of Groovy while reading the parent as well. I always feel like any language that defines itself in the shadow of another language is challenged to find its own true identity. Something I like about Groovy though is that it doesn't have pretenses: it shamelessly tries to be as close to Java as it can. They are busy right now redesigning the syntax so you can write '=>' instead of '->' in lambdas just so it can be more "java-like". I can't think of a more pointless change from a functional point of view but they are doing it anyway. When I use Groovy, I'm not doing it because "Java is bad", I'm using it because "Java is good" but "Groovy is better". When I code in Scala I keep feeling bad every time I divert back to Java-style and constantly struggle to figure out what paradigm is right and when to switch between them.
> When I use Groovy, I'm not doing it because "Java is bad", I'm using it because "Java is good" but "Groovy is better". When I code in Scala I keep feeling bad every time I divert back to Java-style and constantly struggle to figure out what paradigm is right and when to switch between them.
Take it slow, and don't expect to be using every Scala feature/library/pattern from day 1 - there is a lot of depth to functional idioms, the learning curve of something like Haskell has less to do with syntax than you might think. Write code that does things that are useful to you - even if you're only writing "Java without semicolons", you're getting value out of Scala - and adopt the fancier features one step at a time. I've been working in Scala for 8 years and there are still places I'd fear to tread (why do I care if my profunctor has strength?), but every year I've produced more concise, clearer, less buggy code than the year before, and even in year 1 I was doing as good as I had been in Java or Groovy.
Well, "love" is obviously an extreme, but I think I know what DeGoes means. I know I've cursed at Scala's Java/OOP heritage more than once. I've also puzzled over the related issue of how complex some of Scala's function signatures are (or used to be; I know there's work on this) compared to equivalent Haskell functions.
It seems to me DeGoes is arguing that Scala's "key value proposition" is a losing battle, because other languages are starting to do it much better. It's likely that the "practical programmer" will go with Kotlin or even the next iterations of Java, and the rest... well, you can guess the reaction of the audience to his question ("how many of you would work with Haskell if you could?"), and I must take his assertion at face value that in Scala conferences, the more popular talks are about FP, not about OOP or even FP+OOP. Maybe he's wrong about this, of course.
Again, I'm saying all this as a Scala practitioner. I'm not leaving it for Kotlin either. Then again, I certainly would work with Haskell if I could!
edit: one more thing:
> "I've seen lots of people suggest that OOP+FP fusion is what allows Scala to provide an easy gradual on-ramp to FP for people and projects transitioning from Java"
In my experience, this ramping up to FP happens in one of two ways with Scala:
- Completely ad hoc, misunderstanding every principle and making a Frankenstein's monster out of it. No lessons learned, just a mess that "works".
- Understanding it and doing it with care, and mentioning Haskell a lot (as these people start to read articles and watch tech talks on FP).
This is entirely my personal experience and probably won't generalize to every workplace.
Ok, I just want people to not think there are no Haskell jobs out there. The only reason not to be able to get one is if you don't want move and can't find something remote (although there are a few remote jobs around). There are otherwise plenty of Haskell jobs for those that are looking and sometimes in unusual places such as Alaska!
A minor note concerning the usual characterization of the Scala language itself as a fusion of OO and functional, it's definitely not an equal fusion. Not that you proposed that the fusion is equal, but I see it come up elsewhere.
The core language is very much OO. The functional aspects of Scala are either syntax sugar that desugars to OO stuff (e.g. automatic eta expansion, special syntax for defining FunctionN vs new classes, case syntax for partial functions, context bounds) or decisions made by the standard library (e.g. the prevalence of immutable data structures).
Understanding this has been the key for me to understanding where most of the warts of Scala come from. For example some of the weirdness of functions vs methods, weird performance issues, the existence of some of the encoding tricks that libraries like cats and scalaz use, and strange decisions made in the collection libraries all can be fundamentally traced back to this layering.
That being said you can do a lot of FP in Scala because of the combination of ecosystem and syntax sugar, but you have to always keep in mind that you're encoding FP concepts with OO tools (not as equals) otherwise Scala's limitations will come back to bite you.
"(by which I mainly mean: too many people like DeGoes will shout you down rather than trying to help you)."
Proposition for discussion (i.e., I don't necessarily believe this, but I don't necessarily not either): The biggest challenge facing languages and their communities that focus on correctness is that the very personality types that are willing to focus on them are also statistically much more likely to be, ah, potentially less suited to the sorts of compromises and engineering discussions that are necessary to make a personal language than other more pragmatic communities. The sort of relatively uncompromising spirit necessary to swim upstream against the current general consensus of very pragmatic programming nearly entirely unconcerned with academic-type correctness is, well... uncompromising.
If I understood the talk correctly, except for the OOP+FP part, DeGoes seems to be arguing out of practicality, not correctness. He correctly argues, for example, that "FP + OOP in a concise language" is irrelevant as a marketing/business speech for Scala. He seemed to me to actually want to make those compromises and engineering decisions you mention, while maybe admitting that the existing ones are not as successful as initially hoped for.
This is exactly what drove me out of the lisp community. There are just too many people that insist on seeing engineering decisions between different sets of trade offs as stark choices between good and evil. Go read some of Eric Naggum’s rants for a really extreme example of this syndrome.
You're still seething with hostility with regard to the conditions of a Usenet newsgroup fifteen years ago that is only very lightly active now. Reddit has r/Lisp which is more active and very different from old c.l.l. (Not that it's reasonable to equate r/Lisp with "Lisp community", either.)
Partial functions are undefined outside of the part that is given, therefore the result of an input that falls in that zone is undefined. This is not good, how do you deal with it, exceptions? Termination? You must make the function total by extending it to a version that gives some result to manipulate whatever the input (or restrict the domain to make it again total.)
Thanks for that clarification. I had googled for "partial function vs currying", which led me to results for "partial function application", which was really about partial application of functions, not partial functions. :-/
I swear, probably 80% of my problems in professional life are from overloaded terms.
Currying always returns another function (if I’m not mistaken it doesn’t even evaluate anything yet) so it’s another thing and the compiler has a complete understanding of what will happen at runtime. With a partial it won’t be able to do that. The later comment with the Try is a great example
His basic argument hinges on incorrect division of devs into Java/OOP crowd and Haskell/FP crowd. There are plenty of Scala users who don't fit these stereotypes though. I personally very much appreciate the unique mix of OOP and FP that Scala allows for.
For me Scala is a very general and a remarkably unopinionated language. It efficiently encodes and integrates multiple programming paradigms in one language, letting you pick and choose what makes sense to you.
Scala is useful to a wider range of programmer preferences and skill sets than any Haskell-on-the-JVM could ever be. It's a feature, not a bug, both for individuals and enterprises. Pure FP is just some people's preference that requires a significant buy-in, not the one true way of doing things.
In general, this talk just seems like a case for one man's preferences rather than a fair summary of Scala's current market situation.
- Tooling needs improvements. He mentioned how the Scala plugin for IntelliJ IDEA will flag working code as incorrect. It will also fail to flag code that won't compile as written. He didn't mention sbt as a pain point, perhaps because it goes without saying.
- Scala 3 (Dotty) has the potential to split the community and libraries for years, much like the Python 2/3 transition. That said, I'm still kind of excited about Dotty because I only have 10s of thousands of lines of code that potentially need porting work, not 10s of millions, and that seems worth it in exchange for stable macros, faster compilation, etc.
- The Scala community is losing people who just want "better Java" to Kotlin, and losing people who want pure FP to Haskell. I came to Scala because of Spark and I keep using it because I have grown to really like it, despite the pain points. I'm not eager to jump ship toward Kotlin, plain old Java, or Haskell. But if the community keeps shrinking, and the ecosystem shrinks with it, I will eventually be forced to jump toward something else.
I encountered IDEA pain with implicits and type checking just yesterday, relating to a library that uses Cats and Shapeless. It happens enough to remark upon. It doesn't happen often enough to make me wish I were still using Python for data processing.
The Dotty transition looks like it will be significantly more breaking than the changes in the 2.x releases over the past 5 years. I'll be pleasantly surprised if the transition goes smoother than I expect, though.
I am excited by some recent alternatives to sbt, particularly Li Haoyi's Mill.
I did a talk in Portland at their Scala meetup and one of their organizers often takes an open source project and starts hacking on it. Every time he pulls down SBT, he finds it a terribly painful mess.
I've never looked deep into SBT myself, but the sbt-native-packager plugin use to be an insane mess of craziness the last time I looked at it.
As far as tooling, I got pretty use to running sbt even if I got red lines and felt that my types were correct. Often it was to see what the compiler error would be and then I'd be surprised to see it compile with no warnings.
Scala is complex. The Intellij issues an be frustrating, but I can also understand how difficult it is to resolve the complexities in Scala.
I was thinking about Python3/Perl6 as he was talking and glad when he brought it up.
It's important to note, the world has transitioned to Python3. Django removed the py2 comparability imports a while ago and the Python3 Wall of Superpowers stopped updating a while back when it was over 90% green.
For most devs, all new Python code that's green field is Py3. There is still a lot of py2, yes .. a lot in commercial/closed apps that have embed python in there, along with scientific applications. But I think that will change eventually too.
Perl6 is the opposite end of the spectrum. Massive Perl codebases will always be Perl5. I don't see any enterprise people going to Perl6. If you're like Booking.com and still using Perl for everything, you code is probably so decrepit, old and lacking meaningful testing you should just burn your entire company to the ground so your devs are no longer in miserable agony.
He has a point with Python3: it is a large community. Even with that large community, and even with how small many of the syntax changes were, it took a long time to get to py3! Scala is nowhere near as big.
Scala needs to move carefully with Scala3, so that Scala3 is more like Python3 and doesn't end up like Perl6.
> If you're like Booking.com and still using Perl for everything, you code is probably so decrepit, old and lacking meaningful testing you should just burn your entire company to the ground so your devs are no longer in miserable agony.
That's a lot of assumptions:
* Booking.com still using Perl for everything
* Code being decrepit
* Code lacking meaningful testing
* Developers in agony
Do you have some inside information to know that all of these are true?
You also don't seem to understand that companies are in it for the money, not to provide nice new development projects to make developers happy. If you are programming for a living, you should know that programs are written to make money.
By the way, torturing the developers to make a better product, is a well known meme among Perl 6 core developers.
Python3 is/was hard, because it forced some problems to the forefront, that were under the mat with Py2. There was no automatic upgrade path, because what was a silent undefined behaviour in Py2 became an unhandled edge case in Py3. And a lot of very pragmatic people just wanted nothing to do with this, after all, their Py2 program worked for them, so why are now they forced to make it work again in some hypothetical scenarios they don't really care about?
Any maybe that's something Py3 could have handled with a py2 interop (at module load time you set assumptions that lift your py2 code into py3 with those assumptions). But python is not that big on these hacks, it instead just adds 10 more incomplete libraries into its std and calls it progress!
I think this summarizes really well why I never became a Scala developer and am now a Kotlin programmer. I like Kotlin. It solves problems I had as a Java programmer in a way that is easy to understand. It's easy to move from one to the other. Tools are great. Documentation is great. Etc.
Based on this talk, it's not likely I'll ever become a Scala developer.
For reference, I learned FP in the nineties while in university; was exposed to monads, and a few haskell dialects (Gopher mainly). Been doing Java for about 20+ years at this point as well. However, despite this background I have trouble wrapping my head around Scala. The documentation seems to require a phd, which I have, just not in esoteric type system kungfoo. I find most Scala code completely unreadable. I admit, it's me. I'm probably too stupid or something. But if I'm too stupid, then most people I work with are definitely not smart enough either. So, that in a nutshell is why I ended up doing Kotlin instead.
What makes Scala code unreadable is - IMHO - that people rely too much on the IDE to help them understand and navigate code, or their own familiarity with that code (or in some cases the code is simply write only, making it more like obfuscated Perl and KDB+/Q than a modern object-functional hybrid language).
And since Scala has so many faculties, features, facilities to mange code, to abstract problems and their solutions, and people write almost none of their design choices down into READMEs or comments, we're left with the code. For some people it speaks for itself. (For some people types like a -> a tell everything, for me that's just a function with unknown "functionality", because it depends on what the fuck the function actually does. Sure, the type signature constrains the expected return value, and it's useful, but that's not the architecture, not the rationale of/for design choices, it's just the safety striping around the sharp edges.)
Somehow Scala is "the PHP" at the other end of the spectrum. Instead of being too little it usually ends up being too much (insert the relevant Paul Philips rants here), while both are perfectly capable of expressing nicely designed programs/systems.
Furthermore, Scala is actually a bit weak when it comes to expressing really high level concepts. So yes, you can do a lot of magic in it, but most of it ends up being of the Black variety, because "for performance" (minimize GC pressure) or because that part is still "experimental" (macros/reflection) or because that part is just ugly and badly designed (collections).
I agree with many of your points, but on that specific example code you pointed to which I have never used or looked at before, I find it plenty readable if you know how to read it. That doesn’t mean I find it “obvious” if you have never been trained on how to read those signatures or come from another language that does not support these concepts.
Here is how I read this in case it helps anyone
Def zip[A, B](a: ⇒ F[A], b: ⇒ F[B]): F[(A, B)]
F[Something] is a “container type” or a type constructor, a type that is parameterized by another type, for example F[Int] could be List[Int] or Future[Int].
The method zip has two parameters, one a by name reference to something of type F[A] and the other a by name reference to something of type F[B] and returns an F of the pair of input types (A, B)
Looks like this deals with three total types, a container type F that must be the same for input param a, input param b, and the result.
I know nothing about F other than the fact that it contains another type. I know nothing about A and nothing about B which means I can’t be looking them up somewhere or using defaults, so the only way I could possibly return an F[(A, B)] is if I combined the two inputs.
Also, code written in this style is meant to follow the substitution principle, so I’ll think through a couple of substitutions to see how this method could be useful to me.
Since I can’t invent new members of list and the two members of the list may be different lengths, this must return a list whose length is the minimum of a.length and b.length containing and int -> string mapping. Handy.
This can only return me a User if they have logged in at least once. The fact that it is two Futures makes me think I am probably querying two remote servers for this information and that I can’t resolve the new Future until both of the input Futures have resolved. By the time this Future is ready I will have a User with their last login date or the Future will fail.
SBT, implicits, CoC controversy and censorship, scalaz/cats debacle... There are more reasons not to get involved with scala ecosystem than to get involved.
I think people get snippet fever when they see a few lines of really pretty code and they forget how tangled up the scala scene actually is.
I can't believe, after seeing more Rails disasters that I know what to do with, that I'm looking at a software project and thinking "this shit is almost as bad as Rails."
Scala 3 could've been the opportunity to murder sbt to death, abandon implicits entirely, make the breaking changes they need and compose the foundation for 10+ more years of scala hackery. But from what's visible out of dotty currently, it's almost entirely invested in the old sins, with a stronger soundness story nonetheless (congrats DOT team). There's just no way to slow the train now, too much of it is dangling off the bridge and Spiderman won't be able to pull it back.
Good riddance. Thank goodness F# is starting to come together.
What's wrong with (carefully used) implicits? What would you replace them with? Haskell has type-classes, a very similar concept, and it's difficult to imagine contemporary Haskell programming without them.
Implicits are not an afterthought, but a core feature that enables efficient context-dependent
parameterisation. Implicits let you avoid a lot of boiler-plate. They are especially
useful for well-structured ad-hoc polymorphism, quite like Haskell's
type-classes, the predecessor and close relative of implicits.
If you don't like implicits, simply don't use them. You can add
a style checker to your build automation that finds and rejects the
keyword "implicit" in your code. As to use of implicits in libraries,
you are perfectly free to pass explicit arguments instead.
Odersky has a couple of recent youtube videos where he explains
implicits in detail.
Wise words for any project or product that starts in academia but stays there for too long (building for papers and PhDs rather than real customers). Brave to deliver such a cutting critique to this audience too.
As a potential newcomer to Scala, many of these points ring true. Hopefully the community takes a good hard look at itself to see what it can improve.
With some time off this past week I started hunting for things I should be learning. After doing some research on Scala's popularity and trending I came to may of the same conclusions John does.
> building for papers and PhDs rather than real customers
> the community takes a good hard look at itself to see what it can improve
This has already been happening for a while. FYI, the Scala center is entirely dedicated to improving Scala usage and its tooling, which JDG somehow forgets to mention (along with many other positive things happening in the community).
In fact, it turned out in discussions on reddit that JDG is kind of living in his own bubble, not really aware of a lot of things happening outside of it, and thinks he is the only one to see some problems that have in fact been worked on for years. For example, tooling with the Scala center, and the Scala2/Scala3 transition which has been a _central_ consideration in the design and implementation of Dotty since the inception of the project!
Presumably it's a reference to the fact that DeGoes refused to roll over when activists tried to torpedo his conference for inviting the wrong kind of speaker after a double blind process. And continued to be pestered the next year because of another keynote speaker's unrelated tweets.
Having been to Lambdaconf, I can attest to it being an amazing event where people can nerd out shamelessly about functional programming without any bullshit or hostility. I highly recommend it. Easy to forget this is what all of open source used to be like, before the sociopaths moved in.
The supreme irony is the comment above seems to come from a guy kicked out of the Drupal community by the same kind of moralizing busybodies, for being too neuro-atypical and abrasive...
Thank you for diagnosing me as neuo-atypical, some might take offense at that but I don't, I never denied though I never was diagnosed I surely have some sort of mild ASD.
As for being kicked out, that's for the better. Yeah, it took me long enough to get over it but really, those days are gone. I should've left on Jan 8, 2013 when it was made clear how much we've grown apart. I didn't, both the community and I suffered for it. Ending it was good, even if took me close to two years of grieving to get past it.
I fundamentally disagree with calling the CWG "moralizing busybodies", see my take on them https://medium.com/@chx/women-of-drupal-ive-failed-you-and-i... here. What I described there goes for any tech conference organizer: when you are given a chance to amplify a voice, make sure it belongs to someone who deserves their voice amplified. It never was an empty consideration but past #metoo it matters more than ever.
Once you have your eyes open... really, I was watching a pair ice skating on the TV while in the dentist chair (otherwise I wouldn't watch such) and it was appalling how much more skin the clothing of the woman showed than the male. Everywhere we sexualize women. Let me remind you of that time when the South Park creators went to the Oscars in female clothing http://www.trbimg.com/img-535ac5e8/turbine/lat-oscarmoment-l... this?
Rust is great if you want a language with a lot of type/bug safety and want to be able to compile it to run naively on everything from servers to embedded devices.
Scala tries to bring a functional language that can utilize everything that already exists on the JVM, including all the classic Java libraries, while providing a lot of FP idioms, pattern matching, etc.
If you write good Scala code (basically avoid anything and everything written by TypeSafe/LightBend ever .. except for TypeSafe Config .. that one is okay, and it's in Java), you can have some really awesome, clean looking code and programs compared to their Java equivalent boilerplate.
Watched the entire video; still failed to understand why I would pick a JVM interpreted, impure functional programming language over ANSI common LISP which I can compile into machine code?
Which feature(s) is(are) there in Scala which trump running straight machine code for maximum speed and efficiency? I write software which runs on the command line and is designed to take command line options and work well with pipes like all the other UNIX®️ tools and the software has to be efficient on resources and lightning fast. Sell me on Scala.