1. Fix this goddamn 'have to enter 1 extra new line than necessary' bullshit. This is caused by my psuedo_markup function only looking for lines which contain only '\n' and replacing them with '<br>'. This was done initially due to white-space normal being used and many markup-generated html elements ending in newlines. I'll just implement a a sort of hacky fix which ignores newlines if they come after a closing html tag.
2. site-wide image/post expansion. right now expansion only works on the post list pages, by the end of tomorrow it will work on profile pages, and comment pages.
3. I'm finally going to implement some proper caching.
4. Finally, the rest of the PGP functionality. If I can't easily fix the code I have written, how I have written it, I will re-write how it's handled completely from scratching following the MODERN tutorials 1:1. I ran some example openpgpjs code from 2 years ago, which I used as a guide when writing the js pgp functions, and it broke. I'm assuming the library has updated since then, and I will need to change how things work. Worst case scenario, I copy paste a working tutorial pubkey encrypt/decrypt example and wrap it in a function and just use that. The rest of the functionality still works perfectly so I shouldn't have to re-do ALL of it.
5. Caching/Performance. I notice that as record count increased, some areas of the site with large amounts of records are beginning to show they are not cached. For example, somewhat ironically, my user profile probably handles the highest amount of records of any non-cached page on the site currently. Caching will be trivial to implement, it's just a matter of me remember to clearing all the proper cache keys when a new post/comment/vote/whatever occurs. I've left it disabled thus far because it hasn't really affected performance, and a lot of objects are interlinked, so if I update one of them I have to update a few of them. I might also take the time to write caching PROPERLY. An example would be whenever a vote happens. When a post is voted on, it's up/downs changes. When this happens, I have to update the whole object in cache. A way around this would be compartmentalizing the various parts of a post, to where whenever a vote total changes, only some sort of get_votes(cachearg1, cachearg2, etc) needs to be cleared from the cache, instead of the whole object. Or whenever a post is stickied, for example, only some sort of is_stickied() method updates. Also some basic database operation optimization. I can recall, off the top of my head, a few places in the code where a large amounts of records are sorted/whatever unnecessarily, or called necessarily. For example, my user page. Since it only displays recent comments, it SHOULD NOT be getting the entire post/comment history. Me capping this to 15 most recent will provide a significant performance gain, and is something that obviously should be done. It only is not setup this way initially because 1. i was lazy and 2. i wasn't decided on how much information to actually display
That being said, I do not think this would be required. Right now the entire database can be stored in memory with no issues. And the vast majority of the site resources are dedicated to reads, not updates. Even if I implemented caching in the laziest way possible, and just clear the entire cache whenever ANYTHING updates (not going to do this, just making a point) it would still produce a noticeable performance gain as reads outnumber updates by a ratio of nearly 100:1 (thanks bots). I'll most likely just uncomment the existing caching code I have, and simply go 1 step at a time, starting with posts, then comments, then userpages, and modifying every function which updates these objects to also delete the object from cache. I won't be able to cache things necessarily how I'd like without compartmentalizing posts/comments/users a little bit more, but I can cache a fair amount.
I might also just straight up move the filesystem of the site to a ramdisk. Lol. We could scale to many times over the size we are now on a ramdisk, data storage is MINIMAL. Honestly just now thought of this, and it sounds like a meme, but it's a real option. The entire database is like 100mb, the site code/thumbnails is 17mb. The entire linux server is 1.1GB. A large chunk of this size is unnecessary log files and packages. I could legit store the ENTIRE filesystem in memory cheaply well into the near future. I'll probably not do this, as I'd see more-than-good-enough performance gains just by optimizing/caching my code, but it's an entertaining thought.
The being said, it's not like current performance is bad or anything... everything loads pretty quickly even with no caching, and the default werkzkreug wsgi. It might not even be worth the time to focus heavily on this area yet.
My productivity has not been what I'd like it to be over the past few days, mainly because there were issues filling my adderall script lol. We'll be back to normal productivity by tomorrow, assuming the pharmacy doesn't fuck things up again.
Thank you for reading.
Beta Release date tentatively set for Saturday, assuming I put in a few 10+ hour days. Mark your calender boys, we gotta make it count. I will be posting the beta announcement to HN/reddit/some other places. If it achieves even 10% amount of attention it achieved on day 1, we will be in a good spot. Day 1 was literally the first day of public alpha, and the site has come a LONG way since then. user retention will be much higher I feel.
1. Fix this goddamn 'have to enter 1 extra new line than necessary' bullshit. This is caused by my psuedo_markup function only looking for lines which contain only '\n' and replacing them with '<br>'. This was done initially due to white-space normal being used and many markup-generated html elements ending in newlines. I'll just implement a a sort of hacky fix which ignores newlines if they come after a closing html tag.
2. site-wide image/post expansion. right now expansion only works on the post list pages, by the end of tomorrow it will work on profile pages, and comment pages.
3. I'm finally going to implement some proper caching.
4. Finally, the rest of the PGP functionality. If I can't easily fix the code I have written, how I have written it, I will re-write how it's handled completely from scratching following the MODERN tutorials 1:1. I ran some example openpgpjs code from 2 years ago, which I used as a guide when writing the js pgp functions, and it broke. I'm assuming the library has updated since then, and I will need to change how things work. Worst case scenario, I copy paste a working tutorial pubkey encrypt/decrypt example and wrap it in a function and just use that. The rest of the functionality still works perfectly so I shouldn't have to re-do ALL of it.
5. Caching/Performance. I notice that as record count increased, some areas of the site with large amounts of records are beginning to show they are not cached. For example, somewhat ironically, my user profile probably handles the highest amount of records of any non-cached page on the site currently. Caching will be trivial to implement, it's just a matter of me remember to clearing all the proper cache keys when a new post/comment/vote/whatever occurs. I've left it disabled thus far because it hasn't really affected performance, and a lot of objects are interlinked, so if I update one of them I have to update a few of them. I might also take the time to write caching PROPERLY. An example would be whenever a vote happens. When a post is voted on, it's up/downs changes. When this happens, I have to update the whole object in cache. A way around this would be compartmentalizing the various parts of a post, to where whenever a vote total changes, only some sort of get_votes(cachearg1, cachearg2, etc) needs to be cleared from the cache, instead of the whole object. Or whenever a post is stickied, for example, only some sort of is_stickied() method updates. Also some basic database operation optimization. I can recall, off the top of my head, a few places in the code where a large amounts of records are sorted/whatever unnecessarily, or called necessarily. For example, my user page. Since it only displays recent comments, it SHOULD NOT be getting the entire post/comment history. Me capping this to 15 most recent will provide a significant performance gain, and is something that obviously should be done. It only is not setup this way initially because 1. i was lazy and 2. i wasn't decided on how much information to actually display
That being said, I do not think this would be required. Right now the entire database can be stored in memory with no issues. And the vast majority of the site resources are dedicated to reads, not updates. Even if I implemented caching in the laziest way possible, and just clear the entire cache whenever ANYTHING updates (not going to do this, just making a point) it would still produce a noticeable performance gain as reads outnumber updates by a ratio of nearly 100:1 (thanks bots). I'll most likely just uncomment the existing caching code I have, and simply go 1 step at a time, starting with posts, then comments, then userpages, and modifying every function which updates these objects to also delete the object from cache. I won't be able to cache things necessarily how I'd like without compartmentalizing posts/comments/users a little bit more, but I can cache a fair amount.
I might also just straight up move the filesystem of the site to a ramdisk. Lol. We could scale to many times over the size we are now on a ramdisk, data storage is MINIMAL. Honestly just now thought of this, and it sounds like a meme, but it's a real option. The entire database is like 100mb, the site code/thumbnails is 17mb. The entire linux server is 1.1GB. A large chunk of this size is unnecessary log files and packages. I could legit store the ENTIRE filesystem in memory cheaply well into the near future. I'll probably not do this, as I'd see more-than-good-enough performance gains just by optimizing/caching my code, but it's an entertaining thought.
The being said, it's not like current performance is bad or anything... everything loads pretty quickly even with no caching, and the default werkzkreug wsgi. It might not even be worth the time to focus heavily on this area yet.
My productivity has not been what I'd like it to be over the past few days, mainly because there were issues filling my adderall script lol. We'll be back to normal productivity by tomorrow, assuming the pharmacy doesn't fuck things up again.
Thank you for reading.
Beta Release date tentatively set for Saturday, assuming I put in a few 10+ hour days. Mark your calender boys, we gotta make it count. I will be posting the beta announcement to HN/reddit/some other places. If it achieves even 10% amount of attention it achieved on day 1, we will be in a good spot. Day 1 was literally the first day of public alpha, and the site has come a LONG way since then. user retention will be much higher I feel.