Grug Logo

Grug Notes

A simple self-organizing tool for text.

date handling has been gutted. And largely turned over to claude 3.5 Sonnet. Seems good so far. Still some optimizing to do, but not slower and seemingly much more reliable than the rule based methods. Previously it was a combo of chatgpt 3.5 and rule based methods. But it would fail a good number of tests and was a patchwork of if statements. And got quite slow. The new method looks for any date entities on the server, and if any are found, claude is called to intelligently decide the dates.
    let me know if it fails you!
currently experiencing openai api issues/rate limits. May break ai entity identification and search.

The core ai functionality has been gutted. Date recognition is now entirely ruled based with some help from small natural language libraries to determine past or future dates. 
    many tests are good
        need more
    bad with holidays
    llm/chatgpt still responsible for tagging people and companies.
        could use additional testing
    still a lot of room for basic speed improvements

fixed funky scroll on mouse click bug 
templating issue fixed if you had a mention on your home page

grr. weird bug on persistent blocks going blank -- just refresh and it comes back, but annoying. will investigate tonight.
    fixed a server side error (I think this was it)
        update it is not!
            but i understand the problem a little more
            caused by polling an unsaved persistent block...
                gotta love sync problems
                    fixed (I think)

css adjustment for the buttons to the left of the lines. better on mobile, same on desktop.
Some new front end code that helps with more natural scrolling and keeping the text within view -- both on computer and in ios. 

Typo fix on template.
Finally switched to newer embedding model 'text-embedding-3-large' for search functionality. Seemingly better and faster search results already, but still needs tuning.
    fixed ai breakage on new accounts caused by this ~12 hrs later

New templating backend deployed. No visible changes, but slight speed improvement, especially for more complex or long pages.
small bug fix in find/create prompt search. It was broken if the prompt was set to 'generate_today' but not show on this day... if that makes sense. Also now, when a block is already existing on the day, it correctly sets focus to that block. 

small bug fix -- html properly escaped on archived blocks (anything older displayed as plain text for rendering speed)
Search and similar prompts features fixed from the Pinecone vector database issues. 
Vector database service Pinecone is down as their service has gone through some breaking changes. Will take 1-2 days to update. Sorry! This means search is failing and similar prompts. :(
on a prompt page, blocks were listed first, then backlinks. Now they are listed mixed together in order of original block date. Works, but still needs ui and speed optimizations.
fixed a caching bug. if you delete a block on the main home page, the cache now correctly clears. For the past few weeks, it would appear to no be deleted until a full refresh or something else changed.
fixed the crazy bug from yesterday where new line or lines added at the start of an existing note would not save. Weirdest thing ever. Browser would seem to remove new lines unless the html started with a new line.

Basically my template needed to be 


instead of:

crazy right?
new model and prompt deployed for ai/edit voice editing feature. Uses full context of the note -- it's a lot better, but I'm sure it's not perfect. 

Also found a crazy bug where a new line added to the start of an existing note doesn't save. Like wut -- need investigation asap.
Updated copy on landing page.

Small adjustment on revision history so the text is selectable.
brotli added back to landing page
bug fix with similar prompts feature that would cause them to never load -- I think related to django upgrade some weeks back.
More major issues fixed for usernames with capital letters in it. Future, Past, Archive may not have been loading and search issues as well. Grug take long time! But work now. 

Some backend updates. Switched PyMYSQL to mysqlclient. Not sure if it makes a speed difference, but I'd like to think it's an improvement. Also laid groundwork for websockets if needed, though note used for anything yet. 
Ooof, fixed a csrf error on audio recording -- partially broken since april 7th. 
fixed a few endpoints that were no longer getting compressed due to yesterdays upgrades. 
first deploy migrating from Django 3.2 to 5! and gunicorn to uvicorn server! Welcome to modern web dev Grug.

We have server sent events -- it means slightly better device sync. You can try type in the same note on two different devices. 

This was a massive deploy. Hopefully nothing fundamentally broken, but Grug watch closely. 😅 Door open for many more improvements. 🤗
fixed drag and drop blocks that were broken last week from revision history update...
More aggressive caching on home, usually clearing cache on write. Shooting for sub 100ms server response. 

Actually, there were like 4-6 deploys today with various bugfixes of the caching. 
A bunch more orm/sql query optimizations. Less queries on load. And a couple indexes added. And importantly pretty significant improvement on save function. SPEED! yay.

Some adjustments to html to improve load speed.
A few small and subtle query optimizations. Very small speed improvement.
Minor adjustments to caching so stale data is avoided.

Better handling of pasted text and auto adjusting the textarea height.
Speed update! Better preloading and some initial client side caching. Little snappier responses. 😍 wondering what I broke 😅🤓
minimum implementation of edit 'revision history' for each block deployed. Needs work and needs diff formating still. Not functional on mobile yet, but soon!
Small bug fix and reliability improvement to search/ask. 

Alteration and hopefully improve to how known entities are relied on vs chatgpt entity recognition. 

I'm sparse on details here, but changes deployed to production! :)
A bunch of backend work on how backlinks are saved. Backlinks now have a creation date that will persist through saves and minor changes. 
    It's not much of a big deal right now, but I'm hoping it can provide some additional context in the future, to help identify stale tasks for example.

Backlinks are copied to new days on persistent blocks.

Persistent blocks remove lines that are strikethroughed out the next day. That is, if the line is surrounded by ~~, it gets deleted.
    Will be adding this as a user setting soon.

Small front end rendering fix.
oh man, fixed a bad bug. Tags from hashtags and brackets were not properly creating backlinks for a while. 

Shaved 6 queries off the home screen load. Probably not a noticeable speed difference, but it brings joy to my heart.
Encoding issues kinda fixed with latest version of chrome. Everything was rendering as symbols. Now seems ok.
Made the headers sticky on the datatype pages.
such a dumb bug to last this long. Capital letters in usernames no longer break urls.
right clicking on the quick toggle button does the equivalent of ctrl-shift-return -- the default being markdown strike through around the line. Already found a cursor positioning bug, will fix later this week lol.
Sorry! there was a major bug since mid january that made natural language dates no longer link -- or actually they were linked, but it wasn't visible on that day. It should be fixed now.
Right clicking on a link brings up a context menu that allows changing the datatype. Minimum viable features!
    Most useful if AI gets it wrong. 
    Also allows creating autotags. 
    Needs to be in view mode, it doesn't make sense in edit mode.
    Unlinked mentions or backlinks are show at the bottom of prompt pages. WIP.
    There are still some edge cases and bugs related to these features!
Added Inter and Roboto as fonts. 
one line bug fix to prevent possible creation of two unique blocks within the same prompt and date. Which should be only possible if you set a title. In this case, the path was created with a timestamp which shouldn't be possible. 
small bug fix where ai tags could occur within existing manual links or urls
fixed a diff match patch error that caused some sync errors around emoji.
First deploy of quick toggle ui feature that adds the ctrl-return shortcut as a clickable feature on desktop and on mobile.
drag handles moved to top of blocks
Fixed some formatting issues from dark design update, probably a dozen or so little tweaks.

Fixed some issues on date views where 'persistent blocks' wouldn't display.
Darker, but cleaner design deployed. 

Smaller text on mobile.
fixed a bug. If two browsers or devices were open. And the block was brand new, ie never saved, typing in one would not sync to the other with out a refresh. Now it should.
Some tests written, and non-visible improvements/bugfixes to sync. Chasing one last bug where you would lose a few words on save. Grug saw maybe 3x a month. Grug think we got it! But needs battle testing.

Slight alteration to speech to text ai/edit prompt. Handles some situations a little more consistently.

UnboundLocalError on search bug fix. Small font change to make date and prompt filtering more obvious during search.
small ui updates, replaced some emoji with svg

started work on save history ui, incomplete
small bug fix on prompt settings form where the textarea formatting was bad
implemented 'generate on days'
Changed favicons & app icons to new logo.

Draft privacy and terms are online.

Made 👉, ✅ default as autotags on new accounts.
Some optimizations to javascript and stopped loading the archive at bottom of the page on mobile only. It was causing performance issues.

added semantically similar blocks on individual block pages. minimum viable feature. not sure how useful yet.
some small formatting adjustments with new logo

fixed on window resize bug where the textarea would not correctly size

fixed some type error bugs in search

fixed formatting on password reset page
landing page update and logo change!
First deploy with autotags implemented.
Fixed bug introduced on oct 22 where speech to search and speech to find broke. :|
Added links to the day headings under "past" and "future". Changed the permalink style from an anchor emoji to / Mon xx. It's a little weird, but I think I like it better than the emoji. Maybe I'll consider a lone #, we used to do that for permalinks in the old web.

Blocks are unique by the url now. The improvement here is that there can be more than one block in the same prompt on the same day -- if one is given a 'title'. Basically this makes it function like a wiki much more effectively. More work to be done.

It's now possible to create a prompt with only an emoji or with emoji's in it. It will convert to a text string that works in the url. However, it was mainly a bug fix and it only works in the backend. More work to be done on the front end to make it a functional feature. #TODO
Another improvement on block ordering -- new blocks created are always appended in whatever column the prompt last had or default to column 3. The vertical row ordering is unique to each day. Column is universal or at the prompt level.

Fixed funky ios bug where links would need to be touched twice before they would go... 

iOS devices no longer tres to focus on loading -- since they keyboard can't come out because that's the way iOS works. But at least now, links and stuff on Daily Notes or wherever work on loading.

bug fix -- block pages would break if the prompt didn't exist yet. 
found another sync bug! But also may have fixed it today. Added another check to prevent duplicate requests. Optimistic this helps a great deal.

increase the clickable area by 5px around AI generated tag/links. Nice ui improvement!
improved audio record button response speed after you get microphone permission from the user. The trick was removing noice cancelation and auto gain things from MediaRecorder set up. Whisper speech to text is so good, don't think it matters there will be no on device noise reduction.
fixed some accessibility things, and increase contrast for a lot of the ui
some refactoring work on sync. Reduced extra sync requests by slight bit. And now server responds with correct hash when a block doesn't exist. There is/was a bug here where sometimes server would overwrite user that happens to me 1-2x a month. Would lose a few words perhaps from seconds ago. This might fix -- but yet to confirm. :|
few hours focusing on speed -- major one is deferring all javascript. Much improved loading speed. Small improvements and error fixes.
fixed a bug where past & future were not formatting correctly on load
fixed bug where search/ask would break when extra spaces were present
first deploy of ai/edit via audio feature. Experimental... but cool!

bug fix: adding a prompt to automatic speech recognition from the prompt's settings caused an error.

bug fix: legacy code on settings page caused an error when changing automatic speech recognition settings
improved positioning of past and future sections at 2 column width
small error bug fix where the entity recognition wouldn't work right away, it would take a refresh or two, on new accounts.
a new user created a username with a space, and it broke the account. Fixed the validation bug. 
fixed a formatting issue where column 2 (daily notes usually) would overlap today's mentions
it shouldn't matter much in daily use, but refactored and improved caching of embeddings -- basically more efficient :)

very low chance, but also made sure no possibility of duplicates in "similar prompts" on prompt pages.
removed all the prompt settings from the settings page. There is no real need since prompts are drag and drop sortable now.

increase contrast on block and prompt settings. 

Removed a click by direct linking to prompt settings from block settings. Not great, but an improvement.
Improved ordering of blocks when it transitions to 2 col and 1 column from 3. 
Better word wrapping, and fixed a rendering error with long words. We no longer break-words. Just wrap on overflow.

Little bit of javascript refactoring. Fixed the padding/height being too short issue when the page is first loaded.
little javascript refactor, fixed a bug that was calling ai endpoint twice!!! 
Finally implemented drag and drop moving of blocks. This changes the column and ordering at the 'prompt' level so the view will be the same on any day.
Did some refactoring of some llm prompts and test code to make converting singular/plural prompt names more robust. Mostly not noticeable, but feels good to get done!
some ui fixes when there is a pop-over style thing, ie search. Fades out properly and no more pointer affecting things in the background issues.
Some small rendering error bugs fixed when there's punctuation around natural language dates.

Also fixed rendering error where inline style wasn't removed correctly on focus out. Surfaced as overlapping text.

Slight ui improvements on speech to text. On desktop, trigger to get microphone access requires 100 ms mouse over, hopefully fewer accidental triggers. On mobile, got initialization response a little faster -- ie nicer to use.
small ui changes -- home button added at bottom left on mobile. Help button moved to menu.

wider table widths on prompt pages for attributes
bunch of small bug fixes w/css and ui. Moved the signup link to be at bottom.

Fixed pull to refresh -- as it was doing it anywhere on the page.

Adjusted css, so textareas don't interrupt scrolling

Can now have prompt level keywords for speech to text.
deployed mvp of speech to text. This took a while, and did quite a bit of experimentation. 
Cursor placement with multiuser editing is getting pretty good. Patches should get applied and the cursor or selection does the right thing.
Small update to search prompts to consider todays
Added a web manifest, so "add to home" runs as a progressive web app -- meaning the url bar is dropped and it loads from cache more instantly. This is possible now that there is sync functionality. HOWEVER, there is no offline support yet -- but the path is closer.

Some small ui improvements and ui bug fixes with that, more to come.
more work on sync. A few bug fixes so that things should converge between all clients.
More work on sync -- made a pretty significant change. Let's call it *semi realtime sync*. It's not google docs level collaborative, but updates do happen and things mostly converge. Using differential sync. For Grug purposes, it does the job -- and creates a path towards multiuser notebooks and tracking changes over time.
    still an mvp feature
    still need to match cursor position when updates happen
    some odd things happen if actually trying to edit exactly at the same time, but this is very much not typical use case.
improved sync. some changes to optimistic locking. Should see far less false errors
changed the css technique on underlines (uses border now) for ai links -- this was a major source of rendering errors.

improved front end -- fixed some rendering errors on resize and refactored code to be more efficient
fixed a front end error that came up from hidden characters
fix some breakages from open ai updates

change to optimistic locking. No longer relies strictly on the last modified for the locking, but an integer. This should fix the occasional out of sync error that would occur when saves would happen back to back and htmx hadn't return the server last modified time. The save count is updated on the front end, so it should stay in sync and only go out of sync if another client updates it.
On prompt pages, the data fields will update after ai is called. So the page feels live. :) 
search will now redirect to the 'date' view if you only type in a date. Things like aug 5, tomorrow, monday, a year from now, etc should all go direct to that page. 

fixed bug, find/create will add block to the date of the datepage it's currently on, instead of always today
ctrl-shift return keyboard shortcut for wrapping a line with ~~ or other characters. First version. Needs work to apply to selections.
ui improvements, better menu and datatype link formatting for both desktop and mobile. Started using alpine js...

made subscriptions page visible in menu

added ctrl-k short cut to move cursor to search and then find/create prompt and back to default focus if it a third time

bug fix on find/create prompt where search results wouldn't disappear after hitting enter.
Implemented a simple warning when block length is over 3000 tokens. When getting primary entities, text will be truncated after 3200 tokens. Previously was not handled at all. #WIP
Updated the intro text on first login.
Fixed some more issues related to persistent prompts. When an empty persistent prompt was created, the main page would crash without an existing block.

Fixed auto-focus on settings page.

When creating a prompt from a data type page, the data type is set with more "confidence" so it will be recognized.

Fixed a bug or two related issues with "persistent" prompts. It was a significant problem with the Scratch Pad default!!!
    Deleting all notes would result in save failure. Fixed.
    Also removed some old behavior where persistent blocks would get copied to tomorrow.
        Now it should always copy from the past to today.
some small improvements on mobile search formatting. Better, not great yet.
better handling of llm mistakes and manual overrides of entities

bunch of database optimizations

htmx preload! faster.
search fix where natural language dates were not selecting local timezone.
rewrote landing page with updated copy

bug fix on login where settings and sign out links would not show due to timezone issue
made search on prompt pages filter to that prompt only
search/ask grug... mvp feature deployed. This one took a while. Chatgpt based only when you ask a question. All the rest of the time, it presents semantic and keywords search results. When there is one clear best result, it'll be presented up top.