abhibeckert @ abhibeckert @beehaw.org

Posts

0

Comments

321

Joined

2 yr. ago

I’m 99% certain this is wrong

? This is how Postgres stores data, as documents, on the local filesystem:

There are hundreds, even thousands, of documents in a typical Postgres database. And similar for other databases.

But anyway, the other side of the issue is more problematic. Converting relational data to, for example, a HTTP response.

Persisting data as documents would be atrocious for performance.

Yep... it's pretty easy to write a query on a moderately large database that returns 1kb of data and takes five minutes to execute. You won't have that issue if your 1kb is a simple file on disk. It'll read in a millisecond.

I was exclusively talking about how to identify if an image was generated by AI or was a real photo.

These images are being created with open source / free models. Whatever watermark feature the open source code has will simply be removed by the criminal.

Watermarking is like a lock on a door. Keeps honest people honest... which is useful, but it's not going to stop any real criminals.

The article you linked disagrees - they said it pretty well:

Of course, some issues come from the fact that people are trying to use the Relational model where it doesn’t suit their use case. That’s why I prefer a document model instead of a tabular one as the default choice. Most of our applications are more suitable for it, as we’re still moving the regular physical world (so documents) into computers. (Read also more in General strategy for migrating relational data to document-based).

I never joined the NOSQL hype-train so I can't comment on that. However I will point out storing documents on a disk is a very well established and proven approach... and it's even how relational databases work under the hood. They generally persist data on the filesystem as documents.

Where I find relational data really falls over is at the conversion point between relational document representation. That typically happens multiple times in a single operation - for example when I hit the reply button on this comment (I assume, haven't read the source code) this is what will happen:

1. my reply will be sent to the server as a document, in body of a HTTP request
2. beehaw's server will convert that document into relational data (with a considerable performance penalty and large surface are for bugs)
3. PostgreSQL is going to convert that relational data back into a document format and write it to the filesystem (more performance issues, more opportunities for bugs)

And every time the comment is loaded (or sent to other servers in the fediverse) that silly "document to relational to document" translation process is repeated over and over and over.

I'd argue it's better, more efficient, to just store this comment as a document because over and over and over it's going to be needed in that format and anyway you ultimately need to write it to disk as a document.

Yes - you should also have a relational index containing critical metadata in the document. The relationship linking that document to the comment that I replied to. The number of upvotes it has received. Etc Etc... but that should be a secondary database, not the primary one. Things like an individual upvote should also be a document, stored as a file on disk (in the format specified by AcitivtyStreams 2.0).

I wouldn't call that "near" SQL, I'd basically just call it SQL. Nothing wrong with that... SQL is great, and using proper language constructs instead of strings makes it even better... but it's not solving the some problem as an ORM.

For me the article touches on the problem but doesn't actually reveal it.

What I see day in and day out is projects using a relational database to store data that is not suited to a relational database. And you can often get away with that fundamental mistake when you're writing raw SQL queries... but as soon as an ORM is involved you're in for a world of pain (or at least, problems with performance).

OK, but what about reading the data?

I've never had a problem writing to a database with an ORM. The problems happen when you try to read data (quickly and efficiently).

Two issues.

First of all a lot of websites show times in your current timezone. Beehaw for example says you wrote your post "2 days ago". Depending what timezone my browser is in, that could display 3 days or 1 day. When I hover over it, it shows the exact date/time in my timezone. Which is handy.

Second it doesn't actually achieve the intended goal. Even if you use a VPN chances are you've set it to give you a public IP address close to you to have good performance. And if your IP address is in Proton's London datacentre but your timezone is UTC+0 when everyone else is UTC+1... well you stand out like a single black sheep in a flock of white ones and ad networks are absolutely fingerprinting you with that.

there’s no telling if they are routing anything more than the traffic from the browser

Yeah that's the problem "there's no telling". Messing with network settings is opening a can of worms and highly likely to cause problems one day.

Yeah I've seen it happen a couple times on National Route 1 (that's the route of highways that go through every major coastal city in Australia). The mapping data is, unfortunately, not as good as it should be.

I blame our government more than the apps. The government probably notified Apple and Google that the road was closed for bridgeworks or something like that and then and never sent a followup when the work was finished. Eventually they'll figure out out and update the map (there's a feedback feature built into both apps, which will accelerate the process).

... or you could use one of many apps that are based on Open Street Map. It's sort of like Wikipedia for maps, and it works really well on dirt roads that Apple/Google/etc don't bother to map at all. They work fine on tarmac too butt he estimated arrival time is rarely very good since those apps don't collect data on how fast people normally drive though a sharp corner or how heavy the traffic is at 7am on a school day. The good thing is if you notice the map is wrong, you can just fix it. Won't happen very often though, because millions of other people are already doing that. The commercial mapping companies would go bankrupt if they hired as many people as there are volunteering their time for free to edit Open Street Map.

Google lets you know you’re going into bad/shitty reception areas and asks you if you want to pre-load maps

Apple Maps pre-loads maps every time you get directions, and the latest version also encourages you to pre-load large areas (sydney is about half a gigabyte) even if you have perfect reception. It's a free service that doesn't monetise user data or have any ads, so Apple doesn't really want iPhones hitting their servers continuously.

If you've pre-loaded your map, it downloads new data when you charge your phone overnight on wifi and only reaches out to the cloud for traffic data (and even then, it still has "typical" traffic based on time of day/day of the week as part of the offline map).

I always find my Google Maps to be better than my fellow 4x4ers CarPlay maps.

Sure, Apple Maps doesn't do off road mapping at all. But CarPlay works with any (good) third party mapping app, including Google Maps and various dedicated off road mapping apps https://www.coffstrails.com/review-of-navigation-apps-for-4wd-adventures/

Your mates just don't know how to use their phone is all.

It's because DevOps need to be able to do anything, and there are some tasks where a Mac has better software. Also, iPhones have amazing integration with Macs. From copy/paste as if they were the same device to being able to open a dev tools inspector on your computer to debug a page loaded up on your phone to just not even needing to use a phone at all (you can run iPhone apps and websites, on every hardware size and operating system version, on a Mac).

Photographers use real cameras

They are real cameras. Have you seen the photos and videos people create with phones? https://www.youtube.com/watch?v=H8qFTgcRV6w

Yes, there are some situations where an iPhone won't get you a great photo... but that's true of any real camera too. You don't take a several hundred thousand dollar Sony HDC-4800 to a wedding for example. Professionals use the right tool for the situation and these days that is often a phone.

journalists, business men, programmers, don’t type their shit on a 1 inch touch keyboard.

I'm two of those, I type constantly on my phone. Yes - I also have a nice mechanical keyboard and I always use it at my desk, but I'm not always at my desk.

Since at the very least the release of Dell XPS 13 we can have “just-works” machines that run Linux.

Uh. No. I love Linux, use it every day, but you need to know what you're doing.

I’ve worked with devs who refused to run docker on their machines because they said they battery got destroyed

How much power a docker container uses depends on the container. Obviously if the container pegs all eight CPU cores at 100% utilisation then yeah - battery life is going to suck. But with commonplace server side software running in the container I'm able to keep docker running all the time on my Mac and get about 18 hours on battery... and that's with a battery quite a bit smaller (therefore lighter) than the battery in the XPS-13.

Unless you work with iOS development there is no real, practical advantage

I have done iOS development in the past, but these days all of the software I write is for Linux. I think a Mac is the best way to develop Linux software - the Mac window manager is so much better than Gnome or KDE and it has really nice integration with other hardware (for example I'm typing this on a keyboard connected to my desktop Mac, but have the browser window open on a screen connected to my laptop Mac... you can do that on Linux, but it's just two clicks to enable it on a Mac and requires installing/configuring/troubleshooting third party software on Linux)

I have Linux installed on my Mac - and it works perfectly... if it was better I would be using it.

Threads was launched before it was finished to capitalise on one of Musk's more brain dead decisions - and the Threads website was initially almost unusable (it's still missing a lot of things - such as federation with Mastodon).

The requirement to use a phone app, if it still exists, wasn't nefarious, it was just the best they could do without delaying the product launch.

As for "the account they already apparently created"... AFAIK threads runs on Instagram's infrastructure. So if you have an Instagram account then, yeah, "signing up to threads" is basically just enabling threads on your existing account. If you don't have an Instagram account I'm pretty sure they create one when you sign up for Threads (but disable Instagram on your account unless you "sign up" over there).

Pretty sure most of those are not AI generated (yet...).

They pay humans $2 an hour to write a paragraph ten different ways, then mix those with other paragraphs written by other people to create huge "content farms" of sites full of ads.

And they are deliberately shit - because they depend on visitors giving up and deciding to click an ad instead of whatever they came to the site for.

If they block Google, they will likely block DDG an every other search engine.

You'll probably need to be logged in to see anything with rate limits so bots can't crawl the site.

I would question wether "good" and "inexpensive" are possible at all on a wrist tracker. Measuring your heart rate from the wrist is technically difficult - it's just too far from your heart and requires expensive sensors, a large battery, and even then a massive R&D budget (as in hundreds of millions) to get the software algorithm right.

Get yourself a chest strap - those are technically much easier to implement. You'll need to look at your phone to view your heart rate, but it should be accurate unlike most wrist trackers.

Even thousand dollar wrist fitness trackers (like the high end Apple Watch models) are often paired to a cheap chest tracker and those watches generally will trust your chest tracker over their own measurements - because even with billions spent on the best wrist tracker possible they still can't be as good as a $30 chest strap.

Look for one that supports the "ANT" standard. They will allow you to view your heart rate in real time on a variety of other devices (phone, watch, gym equipment, etc). ANT trackers just use bluetooth, so they won't send anything to the cloud (unless you pair them to a phone app that does that).

I've seen that happen when a highway was moved to bypass a town or when a windy section of road was replaced by a straight line cutting through the hills... sometimes the mapping companies don't update the maps as quickly as they should. I've also seen them update the map a month before the new highway is open to the public... shrug

Never seen it on an established road that hasn't changed recently though.

I almost wonder if it’s the triangulation of cell towers (assisted GPS) at this point

No that wouldn't be it.

The way GPS works, it takes a 12 minutes for the satellites to send enough data to calculate your approximate location. And note I said "send" enough data. Your device might not receive the data, especially since GPS is designed for very large (basketball sized) antennas mounted up high on top of a military ship surrounded by open seawater with no buildings, trees, car roof, etc for the radio signal to bounce off.

If your device already knows it's approximate location, then the satellites are continuously sending data to calculate your precise location. So... with cell tower triangulation (or even better, wifi triangulation) to calculate an approximate location then Assisted GPS can get a lock on your exact location in a few seconds as long as you're not surrounded by sky scrapers or in a tunnel.

Even if A-GPS isn't available, your phone will just assume you are near the location you were last time it calculated your location. As long as you haven't just hopped off an interstate flight that should work fine too.

Have you ever used it? Have you experienced this bug yourself?