Grug Notes

another major optimization for voice activity detection. like 10x faster! In practice, this shaves off about 1/10th of a second on all voice transcriptions. :)

voice activity detection redesigned -- should be much more reliable. less false negatives and false positives. This is a big big quality improvement.

finally tested on an android phone. Some fixes deployed last week. Lower nav bar was not showing at times...

adjusted some formatting in the help page. Also made a toggle for pull to refresh, so you can't refresh accidentally while the help drawer is open on mobile. 

Significant improvements to audio and ai/edit features:
    Improved voice activity detection
    Using Gemini Flash 2.0 -- better in every way
    👉 Displays speech recognition, and spinner until it's done
    Alternations to the prompt for improved accuracy
    
👉 
Fireworks.ai added as a backup to Groq for transcription services.

Ask Grug (generative ai search) is faster, switched to Flash 1.5 from gpt4o-mini

menu and audio button positioning regression (from a few days ago) fixed

improvement in the reliability of ai/edit audio feature. Still hunting for bugs, but it's notably better.

cursor handling added while audio recording. Basically, cursor position should be maintained while using audio.

bug fix -- audio ai/edit on an empty or new unsaved block was broken

initial prototype of audio lock feature for longer recordings and of diarization (meeting aka multispeaker transcription). Unfortunately not sure how to solve the problem of ios/pwa falling asleep!!! https://bugs.webkit.org/show_bug.cgi?id=254545

Crazy speed improvement on speech to text. Switched speech to text transcription api to Groq Whisper-3-large. It falls back to OpenAI api (whisper 2) which should make everything a little more reliable with that redundancy. I tried whisper-3-large-turbo, and while faster, doesn't seem quite as accurate. 

Also added voice activity detection. Basically it should prevent the random hallucination or jibberish if you submit empty audio. VAD is shockingly tricky to get right. Not absolutely sure this is right. But testing live!

regression fix from yesterday, affected new accounts for a few hours.

small changes to ai/edit feature. added spinning indicator.

Switched ai/edit feature to Gemini Flash model. It's just faster and seems on par with 3.5 haiku. Small bug fix in that feature too where html entities would sometimes get throw in.

fixed help drawer -- it didn't scroll on desktop and sometimes it was needed.

some backend adjustments and refactoring to embeddings. Don't expect any visible changes from this. 

some very subtle improvements to search

voice ai/edit feature uses claude 3.5 haiku -- much improved!

prompt settings button, clean up -- consistently use svg. 

added mouse over hovers to most all the links lol. after 2 years.

added underscore style markdown parsing for bold and italic

improved the fetching code for csrf. Basically send the request sooner, so there is much less chance of a csrf error. 

Adjusted amount of context llm gets for natural language dates prompt. Should be an improvement in accuracy.

fixed regression, prompt settings were mistakenly hidden in a quest for optimizing loading speed. :|

some caching bugs -- too aggressive caching. Right click on entity to change type slightly more reliable, but not quite 100%.

past and future sections removed from mobile screens. I never used it. More likely to use calendar to change days.

And slight adjustment on past/future layout (nothing notable, just hoping it fixes occasional layout bug).

more prefetching on mouseovers and click on mouse down. FASTER woo

also fixed a bug where the page would sometimes scroll with the quick toggle button on a line.

fixed a rendering bug on iOS when spaces would push to the next line. seriously this one was starting to drive me insane. I hope it's all fixed lol. I will never abandon the <textarea>!

persistent prompt pages now work -- that is the block for today is copied from the previous day it was edited, the same way it does on the home page.

persistent prompts also no longer show up on any future day pages. That doesn't make sense.

added ability to "add to home" for any prompt page. Each prompt page now has it's own manifest.json that allow for this.

If you press and hold the little button to the left of the line, it'll do the same as ctrl-shift-return, ie strike through by default. Great for mobile.

Minimum viable offline support. Edits are stored in local storage and cleared once synced.

Also added a very basic service worker that caches static files. Appears to load faster for those who ‘add to home’ and use on mobile.

error fixed on similar block feature for empty blocks. Inconsequential, but feels good to make errors in the logs go away.

removed any dependencies on gpt 3.5

duplicate backlinks where the context is identical are filtered to only show one -- on the home page at least.

added title text to block settings button

polished the copy button a little more, added quick animation on click and cleaned up the code a little.

minimum viable copy button added. need improvement, but seems to work.

moved save indicator and made it fade out

stripe/subscription sync errors fixed

subscribe banner added back to free accounts. it went missing from a template upgrade regression a while back.

django 5 to 5.1 upgrade

So stoked! There's been a nagging rendering error when a name or company was tagged within a url. Basically random occasions would cause crazy overlapping text during editing. Swapped anchor tag for span during edit mode. Cross our fingers, no more weirdness. It's interesting, chrome seems to treat anchor slightly different. Safari seems to do it better.

slight adjustment and improvement to date tagging prompt.

made some changes to how manifest.json is served -- now fully loaded into memory and served static, hopefully as efficiently as possible. I think it was causing a db hit and contributing to crashing during hacker news traffic. 

csrf error fixed when on prompt page and trying to adjust block title or block settings.

super big rewrite and expansion of help/docs page. Added to menu button on computer view as well, even though it's redundant with bottom left link.

some huge fixes on roam import. More robust and hoping urls correctly load now after the import.

login with google in one click -- instead of it previously going to another page to then click again. Nice quality of life improvement.

bug fix for accounts with a username that started or ended with an underscore

significant fixes with roam research import tool. Handles conflicts and duplicates much better.

additional templates moved to jinja -- faster, but not much of a noticeable change.

small layout adjustments. Better alignment of user name and audio input button.

small fix with caching. it would take a hard refresh for a block to disappear after you delete it. Now it properly clears the cache and it goes away.

small type error in frontend fixed. don't think it was causing any issues.

removed unlinked blocks from datatype pages. it's not needed and saves a db query.

beta of roam research json import feature added to menu

tiny bugfix when archive is empty, error at endpoint for new accounts. 

fixed regression where 'today' could be tagged as a natural language dates since the recent update in date parsing. In general, backlinks should not be created to the day of the note.

Front end bug fix, where sometimes audio buttons would be hidden on mobile after you press record. 

bug fix -- prompts would be missing older blocks due to template mistake.

bug fix -- doesn't wig out (scroll weirdly) trying to make a selection on mobile.

fixed a couple javascript errors on landing page. 

switched person and company entity recognition to gpt4o-mini. search/ask is smarter! :)

some lines of code deleted that were causing ai attributes to not work, ie recognizing phone numbers on existing people wasn't reliable. Still needs work and testing.

slight adjustment to automatic date filtering in search/ask as it was sometimes falsely finding a date

date handling has been gutted. And largely turned over to claude 3.5 Sonnet. Seems good so far. Still some optimizing to do, but not slower and seemingly much more reliable than the rule based methods. Previously it was a combo of chatgpt 3.5 and rule based methods. But it would fail a good number of tests and was a patchwork of if statements. And got quite slow. The new method looks for any date entities on the server, and if any are found, claude is called to intelligently decide the dates.
    let me know if it fails you! keizo@grugnotes.com

currently experiencing openai api issues/rate limits. May break ai entity identification and search.
    fixed

The core ai functionality has been gutted. Date recognition is now entirely ruled based with some help from small natural language libraries to determine past or future dates. 
    many tests are good
        need more
    bad with holidays
    llm/chatgpt still responsible for tagging people and companies.
        could use additional testing
    still a lot of room for basic speed improvements

fixed funky scroll on mouse click bug 

templating issue fixed if you had a mention on your home page

grr. weird bug on persistent blocks going blank -- just refresh and it comes back, but annoying. will investigate tonight.
    fixed a server side error (I think this was it)
        update it is not!
            but i understand the problem a little more
            caused by polling an unsaved persistent block...
                gotta love sync problems
                    fixed (I think)

css adjustment for the buttons to the left of the lines. better on mobile, same on desktop.

Some new front end code that helps with more natural scrolling and keeping the text within view -- both on computer and in ios. 

Typo fix on template.

Finally switched to newer embedding model 'text-embedding-3-large' for search functionality. Seemingly better and faster search results already, but still needs tuning.
    fixed ai breakage on new accounts caused by this ~12 hrs later

New templating backend deployed. No visible changes, but slight speed improvement, especially for more complex or long pages.

small bug fix in find/create prompt search. It was broken if the prompt was set to 'generate_today' but not show on this day... if that makes sense. Also now, when a block is already existing on the day, it correctly sets focus to that block. 

small bug fix -- html properly escaped on archived blocks (anything older displayed as plain text for rendering speed)

Search and similar prompts features fixed from the Pinecone vector database issues. 

Vector database service Pinecone is down as their service has gone through some breaking changes. Will take 1-2 days to update. Sorry! This means search is failing and similar prompts. :(

on a prompt page, blocks were listed first, then backlinks. Now they are listed mixed together in order of original block date. Works, but still needs ui and speed optimizations.

fixed a caching bug. if you delete a block on the main home page, the cache now correctly clears. For the past few weeks, it would appear to no be deleted until a full refresh or something else changed.

fixed the crazy bug from yesterday where new line or lines added at the start of an existing note would not save. Weirdest thing ever. Browser would seem to remove new lines unless the html started with a new line.

Basically my template needed to be 

<textarea>
text</textarea>

instead of:
<textarea>text</textarea>

crazy right?

new model and prompt deployed for ai/edit voice editing feature. Uses full context of the note -- it's a lot better, but I'm sure it's not perfect. 

Also found a crazy bug where a new line added to the start of an existing note doesn't save. Like wut -- need investigation asap.

Updated copy on landing page.

Small adjustment on revision history so the text is selectable.

brotli added back to landing page

bug fix with similar prompts feature that would cause them to never load -- I think related to django upgrade some weeks back.

More major issues fixed for usernames with capital letters in it. Future, Past, Archive may not have been loading and search issues as well. Grug take long time! But work now. 

Some backend updates. Switched PyMYSQL to mysqlclient. Not sure if it makes a speed difference, but I'd like to think it's an improvement. Also laid groundwork for websockets if needed, though note used for anything yet. 

Ooof, fixed a csrf error on audio recording -- partially broken since april 7th. 

fixed a few endpoints that were no longer getting compressed due to yesterdays upgrades. 

first deploy migrating from Django 3.2 to 5! and gunicorn to uvicorn server! Welcome to modern web dev Grug.

We have server sent events -- it means slightly better device sync. You can try type in the same note on two different devices. 

This was a massive deploy. Hopefully nothing fundamentally broken, but Grug watch closely. 😅 Door open for many more improvements. 🤗

fixed drag and drop blocks that were broken last week from revision history update...

More aggressive caching on home, usually clearing cache on write. Shooting for sub 100ms server response. 

Actually, there were like 4-6 deploys today with various bugfixes of the caching. 

A bunch more orm/sql query optimizations. Less queries on load. And a couple indexes added. And importantly pretty significant improvement on save function. SPEED! yay.

Some adjustments to html to improve load speed.

A few small and subtle query optimizations. Very small speed improvement.

Minor adjustments to caching so stale data is avoided.

Better handling of pasted text and auto adjusting the textarea height.

Speed update! Better preloading and some initial client side caching. Little snappier responses. 😍 wondering what I broke 😅🤓

minimum implementation of edit 'revision history' for each block deployed. Needs work and needs diff formating still. Not functional on mobile yet, but soon!

Small bug fix and reliability improvement to search/ask. 

Alteration and hopefully improve to how known entities are relied on vs chatgpt entity recognition. 

I'm sparse on details here, but changes deployed to production! :)

A bunch of backend work on how backlinks are saved. Backlinks now have a creation date that will persist through saves and minor changes. 
    It's not much of a big deal right now, but I'm hoping it can provide some additional context in the future, to help identify stale tasks for example.

Backlinks are copied to new days on persistent blocks.

Persistent blocks remove lines that are strikethroughed out the next day. That is, if the line is surrounded by ~~, it gets deleted.
    Will be adding this as a user setting soon.

Small front end rendering fix.

oh man, fixed a bad bug. Tags from hashtags and brackets were not properly creating backlinks for a while. 

Shaved 6 queries off the home screen load. Probably not a noticeable speed difference, but it brings joy to my heart.

Encoding issues kinda fixed with latest version of chrome. Everything was rendering as symbols. Now seems ok.

Made the headers sticky on the datatype pages.

such a dumb bug to last this long. Capital letters in usernames no longer break urls.

right clicking on the quick toggle button does the equivalent of ctrl-shift-return -- the default being markdown strike through around the line. Already found a cursor positioning bug, will fix later this week lol.

Sorry! there was a major bug since mid january that made natural language dates no longer link -- or actually they were linked, but it wasn't visible on that day. It should be fixed now.

Right clicking on a link brings up a context menu that allows changing the datatype. Minimum viable features!
    Most useful if AI gets it wrong. 
    Also allows creating autotags. 
    Needs to be in view mode, it doesn't make sense in edit mode.
    
    Unlinked mentions or backlinks are show at the bottom of prompt pages. WIP.
    
    There are still some edge cases and bugs related to these features!

Added Inter and Roboto as fonts. 

one line bug fix to prevent possible creation of two unique blocks within the same prompt and date. Which should be only possible if you set a title. In this case, the path was created with a timestamp which shouldn't be possible. 

small bug fix where ai tags could occur within existing manual links or urls

fixed a diff match patch error that caused some sync errors around emoji.

First deploy of quick toggle ui feature that adds the ctrl-return shortcut as a clickable feature on desktop and on mobile.

drag handles moved to top of blocks

Fixed some formatting issues from dark design update, probably a dozen or so little tweaks.

Fixed some issues on date views where 'persistent blocks' wouldn't display.

Darker, but cleaner design deployed. 

Smaller text on mobile.

fixed a bug. If two browsers or devices were open. And the block was brand new, ie never saved, typing in one would not sync to the other with out a refresh. Now it should.

Some tests written, and non-visible improvements/bugfixes to sync. Chasing one last bug where you would lose a few words on save. Grug saw maybe 3x a month. Grug think we got it! But needs battle testing.

Slight alteration to speech to text ai/edit prompt. Handles some situations a little more consistently.

UnboundLocalError on search bug fix. Small font change to make date and prompt filtering more obvious during search.

small ui updates, replaced some emoji with svg

started work on save history ui, incomplete

small bug fix on prompt settings form where the textarea formatting was bad

implemented 'generate on days'

Changed favicons & app icons to new logo.

Draft privacy and terms are online.

Made 👉, ✅ default as autotags on new accounts.

Some optimizations to javascript and stopped loading the archive at bottom of the page on mobile only. It was causing performance issues.

added semantically similar blocks on individual block pages. minimum viable feature. not sure how useful yet.

some small formatting adjustments with new logo

fixed on window resize bug where the textarea would not correctly size

fixed some type error bugs in search

fixed formatting on password reset page

landing page update and logo change!

First deploy with autotags implemented.

Fixed bug introduced on oct 22 where speech to search and speech to find broke. :|

Added links to the day headings under "past" and "future". Changed the permalink style from an anchor emoji to / Mon xx. It's a little weird, but I think I like it better than the emoji. Maybe I'll consider a lone #, we used to do that for permalinks in the old web.

Blocks are unique by the url now. The improvement here is that there can be more than one block in the same prompt on the same day -- if one is given a 'title'. Basically this makes it function like a wiki much more effectively. More work to be done.

It's now possible to create a prompt with only an emoji or with emoji's in it. It will convert to a text string that works in the url. However, it was mainly a bug fix and it only works in the backend. More work to be done on the front end to make it a functional feature. #TODO

Another improvement on block ordering -- new blocks created are always appended in whatever column the prompt last had or default to column 3. The vertical row ordering is unique to each day. Column is universal or at the prompt level.

Fixed funky ios bug where links would need to be touched twice before they would go... 

iOS devices no longer tres to focus on loading -- since they keyboard can't come out because that's the way iOS works. But at least now, links and stuff on Daily Notes or wherever work on loading.

bug fix -- block pages would break if the prompt didn't exist yet. 

found another sync bug! But also may have fixed it today. Added another check to prevent duplicate requests. Optimistic this helps a great deal.

increase the clickable area by 5px around AI generated tag/links. Nice ui improvement!

improved audio record button response speed after you get microphone permission from the user. The trick was removing noice cancelation and auto gain things from MediaRecorder set up. Whisper speech to text is so good, don't think it matters there will be no on device noise reduction.

fixed some accessibility things, and increase contrast for a lot of the ui

some refactoring work on sync. Reduced extra sync requests by slight bit. And now server responds with correct hash when a block doesn't exist. There is/was a bug here where sometimes server would overwrite user that happens to me 1-2x a month. Would lose a few words perhaps from seconds ago. This might fix -- but yet to confirm. :|

few hours focusing on speed -- major one is deferring all javascript. Much improved loading speed. Small improvements and error fixes.

fixed a bug where past & future were not formatting correctly on load

fixed bug where search/ask would break when extra spaces were present

first deploy of ai/edit via audio feature. Experimental... but cool!

bug fix: adding a prompt to automatic speech recognition from the prompt's settings caused an error.

bug fix: legacy code on settings page caused an error when changing automatic speech recognition settings

improved positioning of past and future sections at 2 column width

small error bug fix where the entity recognition wouldn't work right away, it would take a refresh or two, on new accounts.

a new user created a username with a space, and it broke the account. Fixed the validation bug. 

fixed a formatting issue where column 2 (daily notes usually) would overlap today's mentions

it shouldn't matter much in daily use, but refactored and improved caching of embeddings -- basically more efficient :)

very low chance, but also made sure no possibility of duplicates in "similar prompts" on prompt pages.

removed all the prompt settings from the settings page. There is no real need since prompts are drag and drop sortable now.

increase contrast on block and prompt settings. 

Removed a click by direct linking to prompt settings from block settings. Not great, but an improvement.

Improved ordering of blocks when it transitions to 2 col and 1 column from 3. 

Better word wrapping, and fixed a rendering error with long words. We no longer break-words. Just wrap on overflow.

Little bit of javascript refactoring. Fixed the padding/height being too short issue when the page is first loaded.

little javascript refactor, fixed a bug that was calling ai endpoint twice!!! 

Finally implemented drag and drop moving of blocks. This changes the column and ordering at the 'prompt' level so the view will be the same on any day.

Did some refactoring of some llm prompts and test code to make converting singular/plural prompt names more robust. Mostly not noticeable, but feels good to get done!

some ui fixes when there is a pop-over style thing, ie search. Fades out properly and no more pointer affecting things in the background issues.

Some small rendering error bugs fixed when there's punctuation around natural language dates.

Also fixed rendering error where inline style wasn't removed correctly on focus out. Surfaced as overlapping text.

Slight ui improvements on speech to text. On desktop, trigger to get microphone access requires 100 ms mouse over, hopefully fewer accidental triggers. On mobile, got initialization response a little faster -- ie nicer to use.

small ui changes -- home button added at bottom left on mobile. Help button moved to menu.

wider table widths on prompt pages for attributes

bunch of small bug fixes w/css and ui. Moved the signup link to be at bottom.

Fixed pull to refresh -- as it was doing it anywhere on the page.

Adjusted css, so textareas don't interrupt scrolling

Can now have prompt level keywords for speech to text.

deployed mvp of speech to text. This took a while, and did quite a bit of experimentation. 

Cursor placement with multiuser editing is getting pretty good. Patches should get applied and the cursor or selection does the right thing.