Manual Typing is No Fun: Introducing TypeWiz!

A New TypeScript Extension Tool that Automatically Adds Types to Your Code

I’m a big fan of TypeScript. The first time I heard about it was around 2013 (release 0.8), but back then, it wasn’t very mature: after trying it out in one or two smaller projects, I abandoned it. I started using it again around release 1.6 (2015) along with Angular 2, and have had such a great experience with it that I’ve slowly started migrating most of my existing projects to it. I also use it for almost all of my new projects.

(If you want to know more about why you might use TypeScript, and why it’s so great, Victor Savkin wrote a great blog post about it, which I highly recommend.)

There is a catch when using TypeScript, though: what are you supposed to do when you want to add types to a large existing codebase?

Manually adding types to your codebase can be a daunting task, especially given how much room there is for errors. It’s true that in some cases, the types will be obvious by just reading the code. But when I was adding types to a big project I inherited recently, I found myself actually running the code and setting a breakpoint inside a function to figure out the types that were being passed to it.

This gave me an idea: build a tool that does this automatically and much faster, and then call it TypeWiz!

Ta-Da! My friend Pavel ‘PK’ Kaminsky was even kind enough to design a logo!

TypeWiz is a tool that helps you enrich your TypeScript code while saving your precious time by automating the task of adding types to your code.

So for example, given the following code:

function calculate(a, b) { 
  return a + b;
}
calculate(5, 10);

The output of TypeWiz would be:

function calculate(a: number, b: number) { 
  return a + b;
}
calculate(5, 10);

(Note the addition of : number type annotations in the first line.)

In the rest of this post, I will explain more about the background of TypeWiz and how it works, and then show how you can use it in your own projects.

Why do you need TypeWiz?

TypeScript helps you ensure code quality (and also helps you explore your code), but it can only go so far on it’s own: you must provide it with type information to make the most of it.

TypeScript comes with type information for built-in JavaScript and DOM types, so it can easily catch errors like this one:

Math.max([10, 20, 30]) 

Math.max does not accept an array, and thus you will get NaN as a result, which is probably not what you wanted to see as your output.

And TypeScript is pretty good on its own: sometimes it’s smart and can automatically infer missing types. For instance, take this function:

function calculate(a, b) {
  return Math.max(a, b);
}

If you try this yourself, and move your mouse over the word calculate, you will notice TypeScript figured out this function returns a number:

It can do so since it knows Math.max always returns a number.

However, as of today, TypeScript can’t tell the types of the parameters a and b — as you can see in the screenshot above, it simply says any, which means it will not be able to catch the error if you call the above function like:

calculate({}, []);

({} and [] are definitely not valid arguments for Math.max()!)

So, you’ve got to add types to your variables! But as I mentioned above, when migrating existing code-bases into TypeScript, it’s often the case that you need to go manually and add the types.

Manually adding types can be a pretty boring and error-prone task at the best of times. It gets even harder when you encounter code you didn’t write, forcing you to follow complicated logic in order to understand what the function expects.

If you want a “fun” exercise, let’s see if you can figure out just by reading this code the types of the parameters the following function expects:

function(a,b,c,d,e) {
  return d+=c,e=a|b<<d,d<0|a&b<<d&&
    (a=e=parseInt((a|b<<c).toString(d=32).replace(/v/,""),d),
    b=new Date%2?1:3),[a,b,d,e];
}

(Source credit: Binary Tetris by Martin Kleppe)

That’s why I created TypeWiz — I want to make the machine do the hard work for you.

How does TypeWiz work?

TypeWiz works by looking at the values passed to your functions at run-time and learning their types. If you have any kind of automated testing (unit-tests, end-to-end tests) with good code coverage, then you will be able to get good type coverage quickly (this is how I got it to work in my firebase-server project).

But even if you don’t, you’re in luck! You can probably still get pretty good type coverage even if you just run your app and let it do its thing for a few minutes while you manually invoke the app’s main flows.

The good thing about this approach is that you get types that actually reflect the values that your function gets in run-time. In theory, this means that the types you’re adding with TypeWiz mirror reality.

In some cases though, you may get the too narrow a type with manual testing. For instance, consider the following function:

function asBoolean(b) {
  return typeof b === 'string' ? b.toLowerCase() === 'yes' : !!b;
}

If TypeWiz has only seen you calling this function with a string argument, TypeWiz will specify the type of b as string. However, if in some other module you invoke it with a Boolean argument (which is perfectly fine), TypeScript will indicate it as an error. In this case, after verifying that the function will indeed work correctly with a Boolean, you can extend the type of b to be string | boolean, which indicates to TypeScript that the function accepts both options.

So you will sometimes still have some work to do after TypeWiz has run, especially if your code coverage is not very high, but you will still save a ton of time compared to adding in all the types manually.

Real-world Case Study

I ran TypeWiz on one of the projects we are working on at BlackBerry, which didn’t have any test coverage at all. This was a Front End project, and after about 5 minutes of just going to random screens and clicking around, TypeWiz was able to find the types for nearly 1,000 different variables (994, to be exact).

This, in turn, caused 195 errors in our project. Most of them were things like multiplying strings by numbers — which works, as JavaScript coerces the strings to numbers anyway when you multiply them, but this is a bad practice.

Some of the other errors resulted from partial coverage (e.g. a function which could in theory get either a string or a number for the first argument, but in our case was only called with a number), and there was one error that uncovered an actual bug we had missed.

Overall, it took us about two hours to go over all these errors and fix the code/types, and now we have so many more types!

Using TypeWiz on Your Projects

The core of TypeWiz is a library that instruments your TypeScript code to collect type information in run-time, and then applies the types that were collected to your source code. In most cases, however, you don’t need to use the API directly — you can use a wrapper relevant to your use-case.

Currently, there are two ready-made wrappers for you to use: one for Node applications, and another for Webpack.

In this post, I’ll explain how the Node wrapper works in detail, as it’ll serve to give you a sense of how TypeWiz actually works, and give a slightly higher-level explanation about the Webpack wrapper. For full instructions on using the Webpack wrapper, check out the the github page.

Using TypeWiz with Node.js

To use TypeWiz with a Node.js application, install the typewiz-node command line tool from npm:

npm install -g typewiz-node

Then run your code once with typewiz-node:

typewiz-node main.ts

This will run the program, and as soon as it terminates, the source code will be updated with the new types — that’s all it takes!

For example, if you main.ts file contained the following code:

function greet(who) {
  return `Hello, ${who}`;
}
console.log(greet('Uri'));

After running typewiz-node, main.ts will be updated to read:

function greet(who: string) {
  return `Hello, ${who}`;
}
 
console.log(greet('Uri'));

As mentioned above, TypeWiz shines in cases where you have unit tests with high code coverage. If you use the Mocha test runner, you can instruct it to instrument the files with TypeWiz and apply the discovered type information back to your source code automatically by adding
-r typewiz-node/dist/register to your mocha command, e.g.:

mocha -r typewiz-node/dist/register src/**/*.spec.ts

It’s that simple!

TypeWiz and Webpack

Using TypeWiz with Webpack involves a few simple steps, which I explain in the github page. Here, I would just like to point out a few interesting things about how the wrapper works.

The magic begins to happen once you add typewiz-webpack as a loader for your TypeScript files. The TypeWiz loader instruments the source files automatically and adds code to report the values passed to your functions in run-time by calling a special function exposed by the TypewizPlugin.

The TypewizPlugin injects a short snippet of code to each of your generated bundles. This snippet exposes a global function called $_$twiz, that is used by the instrumented code to report the discovered types. It also periodically reports all the discovered type information by posting it to a special HTTP endpoint (/__typewiz_report), which then saves it to a file called
collected-types.json.

After running Webpack with TypeWiz for some time, you can apply the discovered type information to your source code by running the following command (make sure the typewiz CLI is installed first):

typewiz applyTypes collected-types.json

In some cases, you may want to edit collected-types.json and remove some of the discovered types, such as Function or object, or you can also remove them from your sources after running applyTypes as explained above.

You can find a demo project with the complete setup described above here:

https://github.com/urish/typewiz-webpack-demo

After cloning the above project, simply run npm install and the npm start to run the project. The navigate to http://localhost:8080/ and you will see a page with a single button and a text box showing the discovered type info as a JSON object. Click the button to invoke a function in the demo project which will trigger the discovery of additional types:

TypeWiz in action!

Finally, run npm run apply-types and see how main.ts is updated with the newly discovered types!

Using TypeWiz with Large Codebases

TypeWiz may discover a lot of types at once and add them to your source code. Some of these types will result TypeScript errors because of poorly-written code (e.g. multiplying strings by number), because the discovered types were incomplete, or because they actually uncovered a bug.

My colleague Ben Laniado came up with a brilliant approach for dealing with these errors gradually. It provides a simple toggle that will switch all the types introduced by TypeWiz on/off, thus enabling you to deal with these errors a few at a time, and not having to deal with them all in one go. It will also mark which types were discovered by TypeWiz, so you can use it to quickly find all these types and verify them manually.

The trick is defining a new global type, called AutoDiscovered in this example, which equals never, which means it accepts no values:

type AutoDiscovered = never;

Then, we will prepend this type to every discovered type. This can be accomplished by providing aprefix option to the applyTypes method of TypeWiz:

const { applyTypes } = require('typewiz');
applyTypes(require('./collected-types.json'), {
  prefix: 'AutoDiscovered | ',
});

So, if we had a function that receives a number argument, TypeWiz will actually write it as:

function increment(value: AutoDiscovered | number) { ... }

The | operator defines a union type — in this example, a type that is either a number or AutoDiscovered. Since we defined AutoDiscovered to equal never, it actually has no effect here, so the effective type is number, and TypeScript will check the value passed to increment (and yell at you if you pass a string, for instance).

However, you can easily suppress this error at any time by redefining AutoDiscovered to equal any:

type AutoDiscovered = any;

This will change the definition above to read any | number, which is effectively any, or in other words, it’ll disable the type checking for these arguments. Thus, this gives you the ability to switch on/off the type checking for the automatically discovered time at any time, and as you gradually go over your code base, you can then confirm the type or fix the errors and remove the AutoDiscovered marker.

Final Thoughts

I believe good developers must not only master their tools and use them efficiently — they should also know how to extend their tools and make them even more powerful (or how to extend them for fun 🐉). Creating TypeWiz taught me a lot about the internals of TypeScript, how to work with Abstract Syntax Trees (AST), and also gave me some insights to how Node.js require() function is implemented, and how to extend WebPack with plugins and loaders.

I would like to extend my thanks to my friends Yonatan Mevorach and Minko Gechev for teaching me about ASTs that the awesome things you can do with them, and to Vincent Ogloblinsky who gave an awesome demo in NG-BE conf last year, showing how to transform the TypeScript AST into… Belgium’s national hymn! Also, I’d like to thank Amit Zur, who made a good point in his JS-Kongress talk about extending and improving our developer tools. It’s the combination of all the above that inspired me to create TypeWiz.

TypeWiz is still in its infancy — I have many ideas of how it could be extended, and new features that can be added to make it even more useful. But first, I’d like to get some feedback from the community. Please try it, report any issues you encounter, send pull-requests with fixes and new exciting features, share your numbers, and let me know if TypeWiz has also saved you any precious time.

Perhaps it will even inspire you to create even more awesome TypeScript tooling of your own!

Update: If you want to learn how I build TypeWiz, check out my new post -Diving into the Internals of TypeScript: How I Built TypeWiz