A monthly newsletter, mostly on julia internals, digestible for casual observers. A biased, incomplete, editorialized list of what I found interesting this month, with contributions from the community.
“Internals” Fora and Core Repos (Slack/Zulip/Discourse/Github):
Making the creation of Memory an “intrinsic” – a special-cased lower-level operation for which the compiler can know more, thus providing better optimisation. Maybe soon enough small arrays will be allocated on the stack and StaticArrays.jl will be needed less. Original PR and the PR being split in more digestible pieces.
The Julia compiler is now just a normal (standard) library that you can install and develop independently of Julia, greatly improving the ergonomics of development. See the main PR and discussion on slack.
Quite a bit of progress on “compiler plugins”: Few months ago Valentin prepared a draft of how code analyzed by external “abstract interpreters” can be executed within Julia (tools like Enzyme/GPUCompiler/Cassette use various cumbersome workarounds for calling externally analyzed/maniputed/compiled Julia code back into the Julia runtime). He gave a talk about it at JuliaCon. Recently Keno suggested related capabilities stemming from his work on making the compiler just another “independent” standard library. After further considerations, a minimal set of capabilities was distilled into a small PR that permits invoke to accept CodeInstance. invoke is the standard way to hook into the method lookup capabilities – you give it the function you want to call, and the type signature for the method you want to call – thus it provides a neat way to override which method is being called. Now it can also be used to call into code compiled by independent Julia compiler plugins, with different semantics compared to the main compiler (e.g. during development of new compiler versions, or completely independent autodiff or GPU compilers). Check the new docstring for details.
The “pretty printing” infrastructure in Julia is sophisticated but still quite underdocumented. Topics about it have come up often. One new summary was discussed on slack.
This newsletter announced the transition to LLVM 16 only last year. We already are on our way to use LLVM 19, so 16 is going out.
Fun edge cases with parsing function()() end discussed on slack.
Standardizing how and when global = gets lowered, sticking to using setglobal!, leading to fewer special cases in the compiler passes.
disable_new_worlds() can now be called, freezing the world age, stopping invalidation tracking, significantly improving the performance of code loading, as long as you know that the new code does not invalidate anything. At the same time we also got a lot more standardization and cleanup to when world ages actually get incremented.
Maybe nth will be added to Base to generalize first and last.
LinearAlgebra was moved to its own repository (but it is not yet an external independently-versioned standard library)
The base ReentrantLock might get an optional warning message on detecting lock contention.
A potentially beneficial new coupling between inference and optimization, where inference can benefit from type information revealed by optimization.
An interesting discussion and potential changes to promote_rule and promote_type and when to use which. Stemming from an earlier conversation on related antipatterns.
The Julia 1.12 feature freeze is announced – January 8th.
An informative discussion on the lack of non-blocking IO in Julia and what it would take to implement it.
Proposal for significant improvements to Julia’s IO interface. Direct link to the proposal itself.
A dive into the details of constant propagation and when can the compiler figure those out, with a simple example.
Yet another insightful discussion on Julia’s type system: nominal typing vs structural typing/traits.
In search of contributors and new maintainers (specify novice/moderate/expert and internals/domain background necessary):
The Center for Quantum Networks is running many bounties and minigrants on a variety of support packages for quantum information science.
SciML is running a large set of funded small projects – a great opportunity to contribute to the community and be paid for it.
ChainRules, a foundational package to the Julia autodiff effort is looking for a new primary maintainer.
Ecosystem Fora, Maintenance, and Colab Promises (Slack/Zulip/Discourse/Github):
Finch.jl v1 is out! Only the best compiler for sparse and structured array fusion
SymbolicRegression.jl v1 is out! Your favorite tool for discovering symbolic models behind regression problems.
A large family of Docstring translation and explanation tooling (including automation through LLMs) published under the umbrella of DocstringTranslation.jl
Interesting interactive-use tool for importing packages from otherwise independent shared environments: ShareAdd.jl.
The Genie.jl web framework now comes with a GUI editor that makes the creation of apps even easier. They recently ran an app-creation competition.
FunctionChains.jl is a convenient way to iterate functions, while waiting on a decision whether f^n will be adopted in the core language.
MiniObserve.jl a neat hybrid of @show, @log, and statistical summaries, on steroids.
A brief conversation of deepcopy vs copy in Julia and why deepcopy should be avoided unless you know you have a good reason to use it (serialization-like)
Rate limiters and REST API wrappers discussed on slack.
ClassicCiphers.jl: A small package implementing old pedagogical ciphers.
Is it not annoying when PkgEval of your package fails because JET can not compile on nightly.
A neat little demo of Quarto and Typst in Julia.
Jjama3.jl for running LLMs through Julia.
RollingWindowArrays.jl for rolling operations over arrays with an extremely simple view-based API.
Reminder: DimensionalData.jl is a great way to work with arrays with named axes.
Reminder: Z3.jl lets you interface with the Z3 solver.
A full synthesizer built on Julia with several mp3 tracks to highlight it.
Julia Autodiff ecosystem (enough topics for its own category this month):
For floats 0.0 !== -0.0. The various types of equality checks have come up often on various community forums, but this time it is about an exciting interaction between this surprising float inequality and autodiff. Discussed on slack with a reference to this informative manuscript.
Mixed-mode sparse Jacobians at insane speeds available in DifferentiationInterface.jl, discussed on slack
How about finite-differences mode in Enzyme, discussed on slack
Notes from other ecosystems:
Unusual Raku features – does anyone feel inspired
A talk on Float semantics in LLVM (the low-level compiler at the bottom of Julia) and when they deviate from standards and how hardware can be pretty unhelpful.
Soapboxes:
A new official Julia Bluesky account is now set up at bsky.app/profile/julialang.org (discussed on slack)
See also: French community newsletter, community calendar, minutes of triage meetings
You can engage in the discussion of this newsletter on Discourse