Grug Notes

Ooof, fixed a csrf error on audio recording -- partially broken since april 7th. 

fixed a few endpoints that were no longer getting compressed due to yesterdays upgrades. 

first deploy migrating from Django 3.2 to 5! and gunicorn to uvicorn server! Welcome to modern web dev Grug.

We have server sent events -- it means slightly better device sync. You can try type in the same note on two different devices. 

This was a massive deploy. Hopefully nothing fundamentally broken, but Grug watch closely. 😅 Door open for many more improvements. 🤗

fixed drag and drop blocks that were broken last week from revision history update...

More aggressive caching on home, usually clearing cache on write. Shooting for sub 100ms server response. 

Actually, there were like 4-6 deploys today with various bugfixes of the caching. 

A bunch more orm/sql query optimizations. Less queries on load. And a couple indexes added. And importantly pretty significant improvement on save function. SPEED! yay.

Some adjustments to html to improve load speed.

A few small and subtle query optimizations. Very small speed improvement.

Minor adjustments to caching so stale data is avoided.

Better handling of pasted text and auto adjusting the textarea height.

Speed update! Better preloading and some initial client side caching. Little snappier responses. 😍 wondering what I broke 😅🤓

minimum implementation of edit 'revision history' for each block deployed. Needs work and needs diff formating still. Not functional on mobile yet, but soon!

Small bug fix and reliability improvement to search/ask. 

Alteration and hopefully improve to how known entities are relied on vs chatgpt entity recognition. 

I'm sparse on details here, but changes deployed to production! :)

A bunch of backend work on how backlinks are saved. Backlinks now have a creation date that will persist through saves and minor changes. 
    It's not much of a big deal right now, but I'm hoping it can provide some additional context in the future, to help identify stale tasks for example.

Backlinks are copied to new days on persistent blocks.

Persistent blocks remove lines that are strikethroughed out the next day. That is, if the line is surrounded by ~~, it gets deleted.
    Will be adding this as a user setting soon.

Small front end rendering fix.

oh man, fixed a bad bug. Tags from hashtags and brackets were not properly creating backlinks for a while. 

Shaved 6 queries off the home screen load. Probably not a noticeable speed difference, but it brings joy to my heart.

Encoding issues kinda fixed with latest version of chrome. Everything was rendering as symbols. Now seems ok.

Made the headers sticky on the datatype pages.

such a dumb bug to last this long. Capital letters in usernames no longer break urls.

right clicking on the quick toggle button does the equivalent of ctrl-shift-return -- the default being markdown strike through around the line. Already found a cursor positioning bug, will fix later this week lol.

Sorry! there was a major bug since mid january that made natural language dates no longer link -- or actually they were linked, but it wasn't visible on that day. It should be fixed now.

Right clicking on a link brings up a context menu that allows changing the datatype. Minimum viable features!
    Most useful if AI gets it wrong. 
    Also allows creating autotags. 
    Needs to be in view mode, it doesn't make sense in edit mode.
    
    Unlinked mentions or backlinks are show at the bottom of prompt pages. WIP.
    
    There are still some edge cases and bugs related to these features!

Added Inter and Roboto as fonts. 

one line bug fix to prevent possible creation of two unique blocks within the same prompt and date. Which should be only possible if you set a title. In this case, the path was created with a timestamp which shouldn't be possible. 

small bug fix where ai tags could occur within existing manual links or urls

fixed a diff match patch error that caused some sync errors around emoji.

First deploy of quick toggle ui feature that adds the ctrl-return shortcut as a clickable feature on desktop and on mobile.

drag handles moved to top of blocks

Fixed some formatting issues from dark design update, probably a dozen or so little tweaks.

Fixed some issues on date views where 'persistent blocks' wouldn't display.

Darker, but cleaner design deployed. 

Smaller text on mobile.

fixed a bug. If two browsers or devices were open. And the block was brand new, ie never saved, typing in one would not sync to the other with out a refresh. Now it should.

Some tests written, and non-visible improvements/bugfixes to sync. Chasing one last bug where you would lose a few words on save. Grug saw maybe 3x a month. Grug think we got it! But needs battle testing.

Slight alteration to speech to text ai/edit prompt. Handles some situations a little more consistently.

UnboundLocalError on search bug fix. Small font change to make date and prompt filtering more obvious during search.

small ui updates, replaced some emoji with svg

started work on save history ui, incomplete

small bug fix on prompt settings form where the textarea formatting was bad

implemented 'generate on days'

Changed favicons & app icons to new logo.

Draft privacy and terms are online.

Made 👉, ✅ default as autotags on new accounts.

Some optimizations to javascript and stopped loading the archive at bottom of the page on mobile only. It was causing performance issues.

added semantically similar blocks on individual block pages. minimum viable feature. not sure how useful yet.

some small formatting adjustments with new logo

fixed on window resize bug where the textarea would not correctly size

fixed some type error bugs in search

fixed formatting on password reset page

landing page update and logo change!

First deploy with autotags implemented.

Fixed bug introduced on oct 22 where speech to search and speech to find broke. :|

Added links to the day headings under "past" and "future". Changed the permalink style from an anchor emoji to / Mon xx. It's a little weird, but I think I like it better than the emoji. Maybe I'll consider a lone #, we used to do that for permalinks in the old web.

Blocks are unique by the url now. The improvement here is that there can be more than one block in the same prompt on the same day -- if one is given a 'title'. Basically this makes it function like a wiki much more effectively. More work to be done.

It's now possible to create a prompt with only an emoji or with emoji's in it. It will convert to a text string that works in the url. However, it was mainly a bug fix and it only works in the backend. More work to be done on the front end to make it a functional feature. #TODO

Another improvement on block ordering -- new blocks created are always appended in whatever column the prompt last had or default to column 3. The vertical row ordering is unique to each day. Column is universal or at the prompt level.

Fixed funky ios bug where links would need to be touched twice before they would go... 

iOS devices no longer tres to focus on loading -- since they keyboard can't come out because that's the way iOS works. But at least now, links and stuff on Daily Notes or wherever work on loading.

bug fix -- block pages would break if the prompt didn't exist yet. 

found another sync bug! But also may have fixed it today. Added another check to prevent duplicate requests. Optimistic this helps a great deal.

increase the clickable area by 5px around AI generated tag/links. Nice ui improvement!

improved audio record button response speed after you get microphone permission from the user. The trick was removing noice cancelation and auto gain things from MediaRecorder set up. Whisper speech to text is so good, don't think it matters there will be no on device noise reduction.

fixed some accessibility things, and increase contrast for a lot of the ui

some refactoring work on sync. Reduced extra sync requests by slight bit. And now server responds with correct hash when a block doesn't exist. There is/was a bug here where sometimes server would overwrite user that happens to me 1-2x a month. Would lose a few words perhaps from seconds ago. This might fix -- but yet to confirm. :|

few hours focusing on speed -- major one is deferring all javascript. Much improved loading speed. Small improvements and error fixes.

fixed a bug where past & future were not formatting correctly on load

fixed bug where search/ask would break when extra spaces were present

first deploy of ai/edit via audio feature. Experimental... but cool!

bug fix: adding a prompt to automatic speech recognition from the prompt's settings caused an error.

bug fix: legacy code on settings page caused an error when changing automatic speech recognition settings

improved positioning of past and future sections at 2 column width

small error bug fix where the entity recognition wouldn't work right away, it would take a refresh or two, on new accounts.

a new user created a username with a space, and it broke the account. Fixed the validation bug. 

fixed a formatting issue where column 2 (daily notes usually) would overlap today's mentions

it shouldn't matter much in daily use, but refactored and improved caching of embeddings -- basically more efficient :)

very low chance, but also made sure no possibility of duplicates in "similar prompts" on prompt pages.

removed all the prompt settings from the settings page. There is no real need since prompts are drag and drop sortable now.

increase contrast on block and prompt settings. 

Removed a click by direct linking to prompt settings from block settings. Not great, but an improvement.

Improved ordering of blocks when it transitions to 2 col and 1 column from 3. 

Better word wrapping, and fixed a rendering error with long words. We no longer break-words. Just wrap on overflow.

Little bit of javascript refactoring. Fixed the padding/height being too short issue when the page is first loaded.

little javascript refactor, fixed a bug that was calling ai endpoint twice!!! 

Finally implemented drag and drop moving of blocks. This changes the column and ordering at the 'prompt' level so the view will be the same on any day.

Did some refactoring of some llm prompts and test code to make converting singular/plural prompt names more robust. Mostly not noticeable, but feels good to get done!

some ui fixes when there is a pop-over style thing, ie search. Fades out properly and no more pointer affecting things in the background issues.

Some small rendering error bugs fixed when there's punctuation around natural language dates.

Also fixed rendering error where inline style wasn't removed correctly on focus out. Surfaced as overlapping text.

Slight ui improvements on speech to text. On desktop, trigger to get microphone access requires 100 ms mouse over, hopefully fewer accidental triggers. On mobile, got initialization response a little faster -- ie nicer to use.

small ui changes -- home button added at bottom left on mobile. Help button moved to menu.

wider table widths on prompt pages for attributes

bunch of small bug fixes w/css and ui. Moved the signup link to be at bottom.

Fixed pull to refresh -- as it was doing it anywhere on the page.

Adjusted css, so textareas don't interrupt scrolling

Can now have prompt level keywords for speech to text.

deployed mvp of speech to text. This took a while, and did quite a bit of experimentation. 

Cursor placement with multiuser editing is getting pretty good. Patches should get applied and the cursor or selection does the right thing.

Small update to search prompts to consider todays

Added a web manifest, so "add to home" runs as a progressive web app -- meaning the url bar is dropped and it loads from cache more instantly. This is possible now that there is sync functionality. HOWEVER, there is no offline support yet -- but the path is closer.

Some small ui improvements and ui bug fixes with that, more to come.

more work on sync. A few bug fixes so that things should converge between all clients.

More work on sync -- made a pretty significant change. Let's call it *semi realtime sync*. It's not google docs level collaborative, but updates do happen and things mostly converge. Using differential sync. For Grug purposes, it does the job -- and creates a path towards multiuser notebooks and tracking changes over time.
    still an mvp feature
    still need to match cursor position when updates happen
    some odd things happen if actually trying to edit exactly at the same time, but this is very much not typical use case.

improved sync. some changes to optimistic locking. Should see far less false errors

changed the css technique on underlines (uses border now) for ai links -- this was a major source of rendering errors.

improved front end -- fixed some rendering errors on resize and refactored code to be more efficient

fixed a front end error that came up from hidden characters

fix some breakages from open ai updates

change to optimistic locking. No longer relies strictly on the last modified for the locking, but an integer. This should fix the occasional out of sync error that would occur when saves would happen back to back and htmx hadn't return the server last modified time. The save count is updated on the front end, so it should stay in sync and only go out of sync if another client updates it.

On prompt pages, the data fields will update after ai is called. So the page feels live. :) 

search will now redirect to the 'date' view if you only type in a date. Things like aug 5, tomorrow, monday, a year from now, etc should all go direct to that page. 

fixed bug, find/create will add block to the date of the datepage it's currently on, instead of always today

ctrl-shift return keyboard shortcut for wrapping a line with ~~ or other characters. First version. Needs work to apply to selections.

ui improvements, better menu and datatype link formatting for both desktop and mobile. Started using alpine js...

made subscriptions page visible in menu

added ctrl-k short cut to move cursor to search and then find/create prompt and back to default focus if it a third time

bug fix on find/create prompt where search results wouldn't disappear after hitting enter.

Implemented a simple warning when block length is over 3000 tokens. When getting primary entities, text will be truncated after 3200 tokens. Previously was not handled at all. #WIP

Updated the intro text on first login.

Fixed some more issues related to persistent prompts. When an empty persistent prompt was created, the main page would crash without an existing block.

Fixed auto-focus on settings page.

When creating a prompt from a data type page, the data type is set with more "confidence" so it will be recognized.

Fixed a bug or two related issues with "persistent" prompts. It was a significant problem with the Scratch Pad default!!!
    Deleting all notes would result in save failure. Fixed.
    Also removed some old behavior where persistent blocks would get copied to tomorrow.
        Now it should always copy from the past to today.
    

some small improvements on mobile search formatting. Better, not great yet.

better handling of llm mistakes and manual overrides of entities

bunch of database optimizations

htmx preload! faster.

search fix where natural language dates were not selecting local timezone.

rewrote landing page with updated copy

bug fix on login where settings and sign out links would not show due to timezone issue

made search on prompt pages filter to that prompt only

search/ask grug... mvp feature deployed. This one took a while. Chatgpt based only when you ask a question. All the rest of the time, it presents semantic and keywords search results. When there is one clear best result, it'll be presented up top.

some corrections to natural language dates. fixed things like this past wednesday. before it was putting that as next week.

small front end bug fix where tags were still highlighted within a longer word. "Action" within extraction for example.

backend changes with ai prompts to use json instead of custom formatting. aiming to be a little more robust on random output from llm.

css improvements for mobile
    datatypes at bottom of screen
fomatting fixes on block view
bug fix where some blocks weren't show in archive. specifically where the prompt had setting generate in past checked.

fixed tiny front end bug where prompts got replaced with prompt capitalizations. So lower case company, might get capitalized and cause a rendering error.

pretty major deploy! major backend changes
    chatgpt api, makes costs manageable!
        and a lot faster!
    known entities are tagged from database!
        this major, helps ai perform better as well
    quite a few tweaks and improvements on the front end with tag highlighting
        less filter conflicts, as text is tokenized so we can manage the chaos better
            no double tagging -- longer tags go first
    a lot of refactoring

broke the date display in yesterdays deploy. fixed.

on mobile -- made it so the space key tabs in when it's white space before the cursor. seems to work! best feature ever!

removed auto login/signup form, replaced with a simpler login form. Should fix any password manager issues, and prevent accidental account creation.

partial rewrite of filters/rendering front end. Fixes many rendering bugs when tags would interfere with each other. Text is tokenized before tags are highlighted to prevent html from getting jammed up.