Generating Types without climbing a tree [eng]

Hi everyone! I'm Mateo Kalina, the co-founder and CTO of Platformatic. Today, I'm coming online to talk about something you know you like: generating types, or maybe not. I don't know. Have you ever generated types? I think so. Anyway, this is my topic for today. But before we start with that, I need to tell you a little bit about me and my story because it matters for the talk. So, first of all, how many ropes do you need to climb a tree? Now, remember this question, and we'll go back to it at the end. First of all, a little bit about me. I am the Vice Chair of the Node.js Technical Steering Committee of Node.js, a board member of the OpenJS Foundation. I've spent eight and a half years as a consultant focused on Node and created Fastify and Pinoo.

Also, I write every week more or less at node-land.dev, and you can see it and get it anyway. So, cool. Yeah. Some of the stuff that I created is Fastify, a web framework for Node that you should probably be using. If you're not, check it out. It's very useful for building APIs. Anyway, I also wrote a lot of other modules, and last year, in 2021, I got, I don't know, 17 billion downloads per year. And this year, it will probably be around 22 billion. It's a lot of downloads. Okay. So if you really like them, I also have a GitHub sponsor page. So please, for every download you make from me, just post it. Just put one cent there. I would love it. Okay. Even a milli cent. I would love it. Okay. Sorry. I'm joking, of course.

So what's going on here? Well, I have a confession to make. After writing all those modules, I never used TypeScript. Almost. I almost never used TypeScript, at least not before 2022. And what happened in 2022? Well, look, I actually started a company, a developer tools company. And I had to face TypeScript. And what problems did we solve? Well, we wanted to solve the problem of building APIs and improving how teams build APIs. Building APIs is a two-sided problem. You have your backend engineer and your front-end engineer, and they are connected by the API, right? The backend engineer produces the API, and the front-end engineer consumes it. Typically, one writes the server, and the other one needs to write a client for that API. But why is that a problem? Well, it's a big issue because it's a lot of duplicated work, and we'll talk a little bit about that in a second.

The company I founded is called platformatic.dev, and you can check us out. I don't want to spend too much time; I'll probably do a little bit of a demo at the end. What do we want to do? Well, we want to help you move very fast from A to B like you are on a very high-speed train. But then, because you need to reach C, which is your custom app and so on and so forth, we actually want to have the same flexibility of a very, very nice electric SUV. So, look. Check it out. What we are focusing on in this presentation is coding on the client.

Coding the client is actually very important to note because it's a very repetitive problem. Okay? What do you do? Well, I spent a few hours writing a new client for an API. That money is absolutely wasted. So how can we avoid that waste and have less code to maintain? What is the better flow? The developer writes code for the API; they don't need to write docs. They just define it. And the front-end engineer or the other microserver on the consumer end just generates the client. It's way simpler, right? There are way fewer things to build and maintain. So, many of you will say, but there are a lot of things out there, and one of those is tRPC.

And you've probably seen a lot of tRPC out there and shown demos of how amazing it is, how it typesets from the backend to the frontend. It's fantastic. Yes, it is. However, it introduces tight coupling between the client and the server, which is okay if you are working solo. But if two things are maintained by two different teams, this is actually a big issue. Even the creator of tRPC acknowledges all of that. So, anyway, it's great, but it involves tight coupling. TsRest, model rest does the same. Okay. So, it still introduces tight coupling between one and the other. It has some benefits over tRPC. It also has some problems. But, you know, it's another alternative if you are working solo or in a very tiny team. But if you're working across various teams and other people are consuming your APIs and so on and so forth, you might want to do something different.

Well, what do we do then? The nice thing is that tight coupling leads to a monolith that's very hard to break down. The problem with monoliths is they're very hard to maintain long term. So, what do we do with the monoliths? Okay. Well, how can we avoid this problem? How can we keep the separation between the front-end and back-end a little bit? More importantly, how can we create an amazing developer experience without the tight coupling between the client and server? Look, this is the big question because I want a client and a server that are not tightly coupled, that are not connected together so much. But I still want an amazing developer experience. What do we do?

Well, note of color, 80% of developers prefer REST to GraphQL. I don't know why. This is the status. I've asked this question multiple times. It's an unofficial benchmark, an unofficial survey. I should probably run a better survey at some point. But look, this is very important anyway. So... I think it's a good number to know. Sorry about this. And... What do we do? Well, the question now is how can we automatically generate a client for our REST APIs? And also, the client should... The generated code should be minimal and simple. We don't want massive dependencies, right? Because massive dependencies are actually a lot of work.

Now, one of the greatest bits is that you can have OpenAPI... OpenAPI as a way to standardize REST endpoints, which seems... I don't know. Good enough for most cases. So it's a standard. This is fixed. And I can have my server generate it. Okay. So I can generate this automatically. Good. Perfect. But, but, but, but, but... How about the consumer? How can we consume this on the frontend? Well... I had a few requirements here. First, I want the routes to be defined via OpenAPI because mostly I get them for free using Fastify. I wanted something that does not do any data validation at runtime. I did not want to waste any time on the client with the response. Why is that? Well, first of all, if I am running a validator on the client, I am actually shipping a validator. So I have load time, which, you know, a few kilobytes. Yeah, probably matter. Also, I am actually executing some work on the client before rendering. And so I'm still losing a little bit of reactivity and so on and so forth.

However, I can see that at some point we will add something here because it's, you know, there is quite a few... In quite a few cases, the frontend team does not trust the backend team and vice versa. So we... People might really want to do that kind of testing. And we just wanted to use fetch. Okay. So... So I started investigating how to generate those things. Okay. And while my investigations were going on, I started with a very simple function and adding two numbers. And I started looking at... At how to use fetch. And I started using the TypeScript compiler API. And... Now, I can't understand this. Okay. Like, if you look at the right... On the right side, there is... And look, you can... I can probably need to move myself. Here we go. And so let's look at the right side. You can see that we have... We create an identifier. We create another identifier.

And other third identifiers. Then we create a function declaration with all of those things. Okay. It's a lot. Okay. Now, you... We can... You can read it. But it's... It's complex. Right? And then we go and print the output. Okay. Can we do something better than this? Well, this has two problems for me. The first one is this is completely disconnected from the code that we were using to do before. Right? Using before. It also uses a lot of lines to generate a tree. And it... This seems very... A gigantic tree to climb. And I really didn't want to climb any trees for this because I was in a rush and I had no time. And... And so on. And then somebody introduced me to this nice library called CodeBlock Writer, which is a very nice library. It's super tiny. It just doesn't do any validation, any check. And there's no safety whatsoever in using this library. In my code... In the code. Right? But what is the trade-off for this? Okay. Well, the trade-off is that it's homomorphic. Okay. It maintains the relationship between the various parts of our generated code.

So if you look at the function add, you can see that in here it's actually very readable that that's the function add. Whereas in the previous example, it was not readable at all, that was a single function add. It was very complex. So I actually very much prefer this because it's actually way easier to spot problems and see the code that's going to be generated because I see, oh, this is a function add. It's a block. It's a function add with a block. Yeah, I can... I can live with that. Okay. It's something that I can understand. So... And they thought this is my approach. I'm going to use this. I'm not going to use the other API. And this was literally like the lazy options. So what does generating a client do? Well, we take the OpenAPI definitions that you can see on the left.

Okay. And you can see that we have the definition on the left, and then we have... Oh, sorry. We have the definition of the left and we map each path. You see the paths. Okay. With an operation ID to a single method. It's great. And also we take the parameters and responses and map that to things too. Whoa. And it... It has... So when you create this thing, you... If... So if you go back in the paths, you can see that we have used the operation ID. Operation ID is optional in OpenAPI. And therefore, if there is not, we invent one. So save that one for you. So please specify yours. And we... This definition, we split the types. So there is the types and the code. So... Because we might want it to generate JavaScript instead of TypeScript. But you still want to use the types. So we split them into files to begin with. Okay. Which is great. So this is now the time for the demo gods. Now, the demo god needs to be with me. So let's... let's... let's... should I take this one? Let's give it a shot. Okay. So first of all, let's create a new. Oh, yeah. It's going great. Okay. Now, we can just do create platformatic. Okay. And create platformatic creates my platformatic thing. I create a platformatic DB because I just want to create a thing. Oh. Okay. Amazing.

Okay. Sorry about this. And I create my platformatic DB. I call it demo. I use SQLite. I am going to say yes, yes, and yes, and yes, and yes, and yes, and yes. Let's just ask. No and no. Okay. This is going to install npm and do all our things. Okay. What has this created for us? Well, a few little things here and there. It's sorry. It's. Demo. Okay. And here clear. And we, it has created a, an API for us very quickly from a SQL database and it's installing stuff. Our, our database is actually very simple. It's just a very simple movies table. So here we go. It has installed everything. Then he has applied the migration and generated the types. Great. Okay. So what do we do now? Well, now we can do CD demo. And here we go. Then we could do npm start and this starts our project and first it compiles it and so on and so forth. And we actually have our platformmatic DB and you can see here it has a movies and so on and so forth API. We can do a bunch of stuff here. What we are interesting for is this URL at the top. So, so, um, platformmatic clients, front end language. Yes, and we need to copy this thing and paste it here. And now it has downloaded the file and generating a bunch of things for us, which we have the types file, which as a, as a demon before, and as you can see, a lot of them. And then I have my TypeScript thing.

And now I can do NPS TX, TSC. Oh, yeah. Yes, of course. So, make their SSC, and I need to move my API inside SSC. And so we run TSC and this compiles, and now we have our, uh, compile file, which is our common JS file with all the things. And what we can try is, for example, we can try to use this in somewhere else. So we could essentially do, sorry, why it's not okay. Yeah. So we are here. We can go inside and call it client, um, test.ts. And we want to import my, uh, uh, for example, getMovies from API, and then we can sync function run. And here we want to get the movies. Okay. So, and now we call it run. So now we can, and TX TSC. Let's compile this last node, this and test and Ooh, it's it's failing. Why it's failing well, because you have not set our, our, our, our base URL. So we can call setBaseURL here, setBaseURL, and we can put 3042, and this is our base URL and then recompile it. And run again. And this is empty. Cool. So we can even create a movie now. So we have getMovies, setBaseURL, createMovie.

So we could, uh, try to create a movie, uh, createMovie, and we want to set the title test and let's log the movie. And then let's log again, the movies that we had before. Okay, it's compile, and then we can run it again and you can see this is the same and it's working as you would expect to, and you get all the client, uh, done for, uh, essentially for free for you. Uh, you can even do something very nice because if you look at the definitions here, you can see there's a function build and with a URL, so it's actually, it's actually an API that I prefer slightly, but if you're on the front end, you might prefer this just a little bit. So you can just set a global, but I typically prefer to do build and, oh yeah, it's, it's, it's a default export, so you can just use build and now you can say, uh, for example, const client = build('http://localhost:3042').

Okay. And now you could do client.getMovies, client.createMovie. And client.getMovies, which is an API that I prefer very much because I can specify my client, create multiple clients, each one with their own thing. And now I could do, uh, NPX DSC, let's compile, let's test, uh, these up and, you know, it's still working as, as expected. If I write again, I get another one and so on and so forth. Pretty cool, right? Um, so with this, uh, I think the, uh, my time's up and, uh, uh. Uh, you know, hopefully the demo gods were with me, but it's actually very easy when you are, uh, when you are recording anyway. Um, so the question for the audience is, did we climb, did we climb that tree? You know, were we able to generate the types without too much effort? Yeah, I think so. But you know, I leave it to you. Uh, thank you very much for being with me today. Uh, we are platformatic. Try your open source tool at docs.platformatic.dev. And, you know. Thank you for watching. Bye. Bye.