Sq.io: jq for databases and more

407 points53 comments12 hours ago
dewey

Sometimes I wonder if it wouldn't be more efficient for people to just learn SQL instead of trying to build tools or layers on top of it that introduce more complexities and are harder to search for.

show comments
pratio

Though I respect and applaud the effort that went into creating this and successfully releasing it, It has fewer features than duckdb supports at the moment.

Duckdb supports both Postgres, Mysql, SQLite and many other extensions.

Postgres: https://duckdb.org/docs/extensions/postgres

MySQL: https://duckdb.org/docs/extensions/mysql

SQLite: https://duckdb.org/docs/extensions/sqlite

You can try this yourself.

1. Clone this repo and create a postgres container with sample data: https://github.com/TemaDobryyR/simple-postgres-container

2. Install duckdb if you haven't and if you have just access it on the console: https://duckdb.org/docs/installation/index?version=stable&en...

3. Load the postgres extension: INSTALL postgres;LOAD postgres;

4. Connect to the postgres database: ATTACH 'dbname=postgres user=postgres host=127.0.0.1 password=postgres' AS db (TYPE POSTGRES, READ_ONLY);

5. SHOW ALL TABLES;

6. select * from db.public.transactions limit 10;

Trying to access SQL data without using SQL only gets you so fa and you can just use basic sql interface for that.

rout39574

I love JQ. But ... I'd never considered its query language to be particularly admirable. If I want to ask questions of some databases, I don't understand why I'd choose JQ's XPATH-like language to do it.

show comments
rurban

Better would be the reverse. SQL queries over json: octosql.

Summerbud

To be honest, JQ is handy but it's so hard to maintain. I found myself not able to fully read other's JQ related script

jasongill

This is interesting. I wonder if there is anything that does the opposite - takes JSON input and allows you to query it with SQL syntax (which would be more appealing to an old-timer like me)

show comments
dartos

Wow what an expensive domain name.

robertclaus

More tools are always great! Even if it doesn't become the mainstream, it's always great to see people explore new ways of dealing with databases!

show comments
candiddevmike

This is neat but I'm not really seeing anything I can't do with standard SQL and CLI tools like psql. Seems like you'd learn more reusable things using standard SQL too.

show comments
tmountain

Even without a JSON column in Postgres, this is pretty trivial: SELECT jsonb_pretty(to_jsonb(employees)) FROM employees;

hvenev

The demo appears too stateful for me. The real power of `jq` is its reliability and the ability to reason about its behavior, which stateful tools inherently lack.

lionkor

For anyone else wondering; it's written in Go, and it keeps state inside its config file, for example sources (like a db connection string).

varenc

I love sq. It's handy for quickly performing simple operations on DBs and outputting that as CSV or JSON. Though my one wish is that the sq query language (SLQ) supported substring matching like SQL's `... LIKE "SOME_STRING%"`. Though you can just invoke SQL manually with `sq sql`

wreq2luz

I was reading about something like json output coming to Postgres one day (https://www.postgresql.org/message-id/flat/ZYBdnGW0gKxXL5I_@...). Also the `.wrangle | .data` wraps on an iPhone 13 mini.

tgmatt

Sorry but I am pronouncing that as 'ess-cue` and there is nothing anyone can do about it.

Looks kinda neat for when I don't want or need anything more than bash for a script.

mlhpdx

Dang, I wish I had this while I still had SQL databases.

renewiltord

Related is Google’s pipe syntax for SQL https://research.google/pubs/sql-has-problems-we-can-fix-the...

mynameyeff

Wow, very cool. I was looking for something like this

mrbluecoat

TSV support might be nice for Zeek logs

fforflo

I love the idea of pushing JQ and other DSLs close to the database. I've written jq extensions for SQLite [0] and Postgres [1], but my approach involves basically embedding=pushing the jq compiler into the db. So you can do `select jq(json, jqprogram)` as an alternative to jsonpath.

Trying to understand: Is the main purpose of this to use jq-syntax for cataloging-like functionality and/or cross-query? I mean it's quite a few lines of code, but you inspect the database catalogs and offer a layer on top of that? I mean, how much data is actually leaving the database?

[0] https://github.com/Florents-Tselai/liteJQ [1] https://github.com/Florents-Tselai/pgJQ

cassepipe

Not to be confused with the gpg alternative from sequoia-pgp also called sq : https://sequoia-pgp.org/

doctorpangloss

At some point, why not package Python into a single executable, and symbolic link applications and modules into it for Unixy-ness?

Another POV is all the developers I know who thrive the most and have found the most success: they rate aesthetic concerns the lowest when looking at their tools. That is to say that the packaging or aesthetic coherence in some broader philosophy matters less than other factors.

show comments