Tell HN: Learn about the state-of-the-art in Program Synthesis

Hi HN: Program synthesis generates code from specifications: test-cases, I-O examples, and assertions. On September 19-20, synthesis experts will speak about recent breakthroughs and practical tools. The talks will be aimed at software engineers who haven’t encountered synthesis before. We want to extend an invitation to the HN community because the periodic posts on program synthesis and Z3 show interest in the technology.

Speakers include researchers from Stanford, Berkeley, UW, MIT, and Google Brain. As a preview, here are 3 topics that will be covered: hw-exploit synthesis (https://www.cs.princeton.edu/~ctrippel/#publications), end-user web programming (https://schasins.com/papers/), verification and synthesis of OS level code (https://unsat.cs.washington.edu/projects/).

Technical abstracts and registration is on: https://synthetic-minds.com/pages/conference/2019/#program. After the conference we’ll post the slides of the talks.

The foundations part will explain why synthesis reduces to solving an \exists\forall query. More specifically, the query asks “does there exists a program P such that for all inputs x, P(x) computes the correct output?” A solution to the query is the synthesized program P. How to solve the query? The Z3 prover works well for boolean satisfiability (i.e., one \exists query). To solve an \exists\forall query, one approach is to have a two Z3 solvers communicate: one solver synthesizes a candidate program P’ that is correct on a sample of inputs, while the other verifies that P’ is correct on all inputs. If P’s is incorrect, a counterexample input is added to the sample on inputs. The two solvers iterate until the latter is satisfied with the correctness. This process is not unlike GANs; it is called CEGIS (counterexample-guided inductive synthesis) and was invented 2005, and built on an earlier CEGAR (counterexample-guided abstraction refinement) technique from the 90s.

Please comment; or email us with questions.

24 points | by saurabh20n 1707 days ago

4 comments

  • saurabh20n 1707 days ago
    I am the OP. Along with Ras and Henele, we are leading efforts in program synthesis at Synthetic Minds (YC S’18). I started working on synthesis in 2007; and Ras has been developing these techniques since the late 90s and is an ACM fellow for his work in program synthesis.

    The researchers will talk about their peer-reviewed work in web automation, hardware security, operating system extensions, programming for non-programmers, automatic code translation, and superoptimization. Hopefully, this will illustrate the power and limitations. We'd love for people to extrapolate from these onto their own domain-specific automation needs.

    By touching upon foundational techniques (making imperative code functional, symbolic compilation, SMT encodings, partial evaluation), hopefully the leap to "code synthesis" will seem less like magic and more like an obvious next step. In addition, open-source frameworks exist (e.g., Rosette, Sketch) that abstract away these foundations, and the program will cover those in hands-on workshops.

    We’d love to hear insights into application from people for whom synthesis is new. Some problems are exciting to us (Synthetic Minds is working on smart contract synthesis); and we’d love for the community to brainstorm applications to their domains.

  • kyboren 1706 days ago
    Cool! Very similar techniques can be used in the hardware domain. For these types of problems, at least, I have found SRI's yices solver to be much, much more performant than Armando's CEGIS solver. Do you have any tips on other solvers you've found to be faster for exists-forall problems?

    Here's a raw example of a 32-bit adder super-optimized according to rough delay, gate complexity, and wire complexity models embedded in the 2QBF: https://share.riseup.net/#xg2ySn41zmhtHtmrlQWn5Q . According to post-PAR results from Innovus, it can actually (barely) beat DesignWare adders at some points in the trade-off space.

    [-]
    • rbodik 1703 days ago
      Hi, this looks like a strong result. Congratulations!

      Short answer: to scale up hardware synthesis, it may be necessary to change the encoding rather than look for a better solver.

      More details: I should first say that I have limited experience with synthesizing hardware. My lessons come from [1], where synthesized a small Wallace Tree multiplier. What that taught me is that hardware arithmetic circuits should perhaps not be formulated as a 2QBF problem -- because you might need too many counterexample inputs to terminate the CEGIS loop.

      Instead, I believe that the circuit synthesis should use algebraic reasoning (for correctness) and combinatorial reasoning to explore the space of candidate circuits. Since one symbolic input is sufficient to show correctness, the problem simplifies from 2QBF to SAT.

      This idea is briefly explained in Sec 4.1 in [3], a project that synthesizes software expressions that look very much like hardware circuits (permutations and such) [2].

      I am happy to discuss this in person if it might be helpful to your work. Both Mangpo (the author of [2,3]) and I will be at the conference.

      [1] https://ieeexplore.ieee.org/document/5227085

      [2] https://github.com/mangpo/swizzle-inventor

      [3] https://github.com/mangpo/mangpo.github.io/blob/master/paper...,

  • AthLado 1707 days ago
    This is very interesting - my eyes stuck on the programming for non-programmers as we in MOOCTORS (https://www.mooctors.com) we are building a tutoring platform for MOOC students who are not necessarily programming experts. We invite technology experts and everyone who can code to register as tutors on their free time, help students on their MOOC courses, and earn an extra income for doing that. How Synthesis could power tech/coding tutoring?

    best, Athanasios

    [-]