Improved Externs Inference
In my previous post I talked about Externs and the options shadow-cljs provides to help deal with them. I sort of skipped over the whole Externs Inference subject since I didn’t feel it was “ready” yet. It worked reasonably well but also generated a whole bunch of warnings that weren’t actually issues (eg. any deftype, defrecord, …). It also was way more ambitious in trying to actually generate Typed Externs.
Typed or Untyped?
I took the work Maria Geller and David Nolen had done as a starting point but decided against using it since I think we can make this process a whole lot easier without sacrificing anything.
The Closure Compiler acts as a type checker and the Closure Library is fully typed, so naturally every carefully crafted externs file you find in the wild is also fully typed. This is great if all your code is typed and the externs code just flows through. However ClojureScript is not typed, therefore you gain nothing by using typed externs.
Whenever the Closure Compiler cannot determine the type of something it will do the “safe” thing and neither remove or rename a property if it is defined anywhere in the externs, regardless of type. For CLJS code this will be what happens 99% of the time.
Untyped FTW!
A lot of CLJSJS packages already cheated and used generated externs that are sort of half-in/half-out. The trouble with that is that it may list too many properties and may actually keep more code alive than required. In the very least it will stop the Closure Compiler from renaming a few things that it might otherwise rename. We only need externs for the code we actually call from CLJS, not everything the JS code uses.
Luckily for us the impact of too many externs is rather minuscule and something we could tweak later if we wanted. It is a much better experience to not run into the dreaded ln.S is not a function errors and the effect on code size is surprisingly small. 3KB gain with generated externs over manually written externs for a 600KB .js file. I think that is very acceptable already but we can get that do almost zero difference by using :infer-externs to actually only generate the things we need.
How does it work?
The official Externs Inference Guide has a good overview and pretty much all of it still applies. However as explained above we do not need to worry about the type.
As the example explains this code
(set! *warn-on-infer* true)
(defn wrap-baz [x]
(.baz x))
would produce this warning
------ WARNING #1 --------------------------------------------------------------
File: ~/project/src/demo/thing.cljs:23:3
--------------------------------------------------------------------------------
19 |
20 | (set! *warn-on-infer* true)
21 |
22 | (defn wrap-baz [x]
23 | (.baz x))
---------^----------------------------------------------------------------------
Cannot infer target type in expression (. x baz)
--------------------------------------------------------------------------------
The guide tells you to add this tag metadata
(defn wrap-baz [^js/Foo.Bar x]
(.baz x))
which we can simplify down to just ^js.
(defn wrap-baz [^js x]
(.baz x))
By using the ^js tag metadata we are just telling the compiler that this is a native JS value and we are going to need externs for it. In shadow-cljs the /Foo.Bar type info is optional. You can still use it if you like but shadow-cljs won’t bother you if you don’t.
I also did a whole sweep through the compiler to get rid of some warnings we don’t actually need externs for. It will now also account for all properties that are actually defined in externs and won’t warn about those.
There are still a few cases left where the compiler might produce a warning when in fact no externs are needed. This will happen if you do native interop on either CLJS or Closure JS objects. We do not need externs here and you can get rid of the warning by using either the ^clj tag for CLJS or the ^goog tag. If you know the actual type of the thing you can also use that instead.
How to use it
By default you still need to (set! *warn-on-infer* true) to actually get warnings about Externs Inference for each individual file.
Since that is a bit tedious to do for every file in your project I introduced a new :compiler-options {:infer-externs :auto} setting (in addition to just true|false). This will turn on the Extern Inference warnings for your files, ie. not files in .jars.
You can try :all but that is crazy talk. There are still about 200 warnings in cljs.core but we don’t need externs for any of them, so yeah do not use :all. Only if you really want to see a whole lot of warnings, I got to over 1200 before all the tweaks. ;)
Also make sure you are using at least shadow-cljs@2.0.61.
Disclaimer
This is not official work. It is an experiment. It is based on my experience with the Closure Compiler and mostly based on some more or less educated guesses. The whole subject is not documented very well and I mostly went through a whole bunch of Java Sources to figure it out. It works well for me but YMMV. It definitely could use a few more real world examples to test this on.
If everything works out the way I hope we can make this official and part of the CLJS itself. Until then it is shadow-cljs only.
I hang out in the Clojurians Slack if you have questions. Feel free to open a Github Issue if you run into trouble as well.