Hey, hello everybody, and welcome to another episode of Philip talking into the void.
I thought I'd use this walk today to give you a little bit of an update of what has happened since our call on Monday, summarize it a little bit for Bee, and also kind of just put my thinking into words of how to move forward and how to launch like a nice product for that.
So, maybe to bring Bee up to speed, first on Monday, basically Valerie and Tom and me sat together, talked a little bit about like this vision of building an extension.
And it get- I guess it became increasingly clear that the- this team is so well-positioned to build like a completion thing that it's almost like stupid to think about anything else.
So, prior to the call, I set up like a very, very basic extension scaffold. Started working a little bit on like a website. Didn't really know what to do. I was using this call mostly to figure out like what the major building blocks are and how we distribute our time.
And I think one pretty big discussion point that Valerie brought up is that we should all like...
we need a way to keep ourselves a little bit accountable so that we can actually find the time and work on this open source project that we'd all wanted to do, to avoid basically like a Vibe Insights 2.0 disaster where we're just no longer motivated.
And it's- it's funny because for me motivation is like a sprint.
I feel super excited about something, and then I'm gonna work like every like free hour I have and even some work hours, whatever, (laughs) on that thing, only eventually to like slowly get burned out.
And that's like the point where I really need someone else to like continue pushing so I don't... keep the motivation.
Personally, I'd love to have something that we feel good about shipping, at least like sharing to other people this year still.
I know this is a bit brutal, specifically because of the- some of the like technical challenges we have. Uh, I guess the biggest challenge we have is that we don't really have a good cloud offering.
We cannot afford $250 per day, or at least don't want to afford that right now with no traction.
Doesn't seem useful, and that's like the- the price range for running the, like, smallest possible cluster for client friends. Uh, so that's something I've been thinking of a little bit over the past week, past few days too.
And I stumbled upon this model that, uh, the folks at Kontinua.dev have been creating. As far as I understand, it's also a fine tune of quant 2.5 whatever, blah, blah, 7B.
It's pretty similar to Zeta, the one from Zed, but it seems to be better according to their message, whatever that even means.
I don't think it needs to be particularly good, but the fe- feature that this thing has is someone from the Kontinua.dev team thought about putting this thing on Ollama. They had like, they added like a four, five-bit quantization, blah, blah, blah.
I don't know what these like abbreviations even mean, but basically you can run it. I ran it. I ran it (laughs). I get, uh, I tested it out already today, and I was able to like...
the one, the context window was pretty s- small. You probably wanna feed it more information.
But I was able to get, uh, in fair, like the next edit base in like a smaller editable area in what? Like less than a second.
So it was pretty, pretty usable actually.
Um, so yeah, I think that could be really good for us because the people that I think would want like an alternative to, of course, a tab, there's like, of course there's people that want like cursor tab quality in their IDE, which is something that we just need like more time, uh, and specifically also more data.
But there's also a crowd that just wants to like not have their stuff in the cloud. They want like an open source service because they're just a little bit more schizophrenic about their data. And I feel like we can totally capture that point.
So that's like the first thing, um, I wanna do, like figure out edit.
So the edit stuff itself, we figured that's better for Valerie and Tom to take a look at.
So they've been... they'll drive the implementation side of the extension.
I might get to start a little bit looking at copying stuff from the Kodi extension, but mainly because I like really wanna like wire things up and get in a done state.
And I feel I'm just a person that has like the most time to spend on this right now, interestingly, which by the way shouldn't be like any judgment or whatever. It's- I guess, again, it's my hype cycle works like that.
I need to do the best thing of my super motivation time, so when my non-motivated time comes, you like are set up for success. The other thing we should ask is a little bit of architecture. I already made a lot of changes to the existing repo that everybody has access to.
It's in the Diffs.io GitHub org.
Um, there is now an LSP package that implements a normal language server, and if you look it up actually, GitHub Copilot does have a language server as well, which is not really documented but like people are using it, and it's kind of like some APIs are documented.
And I just kind of want to build things the same way because then it's like very clear for how to do some of those things, and it gives us a lot of things for free, like the whole document processing and the whole edit tracking wiring just works out of the box because LSP has protocols for that, right?
And even like text inline completions actually are in the LSP spec now. They just don't work like this in VS Code, but they do edit us and you can just hook this up probably with, maybe with Zed.
We can take a look later.But yeah, for edit, next edit, this thing is still pretty experimental and we'll have to, like, build our own custom extension to the LSP server, custom messages.
But it's actually pretty nice and you can even get it typed to some extent.
So the current setup works pretty well for, like, inline completions. I have, like, very, like, simple setup with, like, an API key to this Inception AI, like, fill-in-the-middle provider, whatever, that works.
And I want to focus a little bit more on the onboarding experience next, because I think what we should make thing- ma- make clear when people try this out is that we can only be successful if people share data with us.
So I've been thinking of trying of tr- three tiers, uh, of data collection, and I think when you first start open code in your editor, it's gonna show like maybe... I don't want to have, like, any fancy UI and HTML.
I don't think we need that, but if the bottom or so says, "Open tab," and it's, like, disabled with a yellow background or whatever, and you click it, it gives you, like, a kind of an onboarding experience through the, the Quick Picker or whatever it's called, command palette stuff, where the first thing is, like, select your, like, data retention thing.
And the pre-selected stuff will be share when working in open source, which I think is a pretty good default, and we can, we just, like, look at the Git remote if it contains GitHub, make a fetch request with an authentication.
If that is authenticate- if that gives you data, it's, like, open source and we, we will collect data. I think this check should also be (1:30:00), but whatever, not the point. And then you can also, like, of course disable and share everything for, you know, us.
We really need to get data going and I think we should make it clear that this is the only way we'll be... There will ever be a better cursive type is when people kind of install the thing and trust us with some of the data.
Yeah, the next thing I think in this onboarding experience is gonna be, like, which inference provider or like which completion provider they use. Uh, so I think the local one with Olama and this thing could work. Maybe it's gonna be slow. We can even explain a little slower.
There should ideally be the OpenType Cloud, which you, you know, you have our, like, hosted stuff. We just don't really have a hosted stuff right now. We can, we can put Inception AI stuff in a, in, like, wrapped around it, we'll see.
And then maybe even some bring your own key stuff.
I saw that some NeoVim extensions, I forgot the name on top of my head, but I can send it to you if you ever were to ask me, that some NeoVim extensions do, uh, (1:31:05) prompt, like Haiku 4.5 with, like, a specific prompt and it's like, you know, these models have been done pretty good.
So maybe this is something at least, you know, the average person would want to use it and don't want to spend money could install it. I don't think we necessarily have to be... Have to make money off this right away.
If we get traction and this thing has, like, a thousand downloads overnight, (laughs) you know, just dreaming, right?
But if we do, we'll be in, like, a much better position to figure out what we actually want to do. And some of the discussions we had very early in this group might actually become relevant.
But until then, let's just build something that is, like, good, like, good enough to be in our workflows and, like, explain the vision we have, which I think also means we want, like, the VS Code extension and probably something else.
I think if you do, like, a NeoVim extension and there are some plug- plugins out there, it would be awesome because the people, like, they're craving for something like this.
(sniffs) By the way, I have only mentioned this briefly, but I was trying to figure out why SuperHaven was so fast and apparently they were not using a transformer for their thing. So, lots of opportunities still.
But yeah, for us right now, I think not necessarily feasible. At least not today. You know, if we have money, everything can be changed. Uh, yeah. If there anything else I wanted to mention?
Not really.
So yeah, hope you like this episode of Philipp talking into the void with the upgraded microphone and curious as- to just see, like, what you guys are thinking, of course, and also how you are thinking about this current setup, if you ever want to try it out.
I think it's okay. Could probably be a lot better. I can probably do another episode just about, like, talking more in depth of this, but it doesn't seem useful right now.
So yeah, it's the biggest thing to point you to bring up to date is I think we've been thinking of meeting kind of more regularly again and really keeping ourselves accountable to push on the thing a little bit.
I would love to have something out there that is usable enough so that we can share this and make it public and figure out if people actually care about this. That'd be awesome.
I never, never shipped any of my side projects and I'm also a little bit scared, but, uh, I think it could be cool.
So anyway, enjoy your rest of the day. Bye.