# orderedItems[].object.content
🥳 HighlanderPG 1.0.6 is out! This adds support for the `:repo` option, allowing easier than ever setup.

HighlanderPG helps you run a singleton process in your cluster, and is available for $100 a year at https://hex.codecodeship.com/package/h ighlander_pg

#elixirlang #myelixirstatus

# orderedItems[].object.contentMap.en
🥳 HighlanderPG 1.0.6 is out! This adds support for the `:repo` option, allowing easier than ever setup.

HighlanderPG helps you run a singleton process in your cluster, and is available for $100 a year at https://hex.codecodeship.com/package/h ighlander_pg

#elixirlang #myelixirstatus

# orderedItems[].object.attachment[].name
code: # lib/application.ex

children = [
  ...
  {HighlanderPG, [child: MyChild, repo: MyApp.Repo]},
  ...
]

Supervisor.init(children, strategy: :one_for_one)

# orderedItems[].object.content
Long form announcement of CurlReq, which became a kind of love letter to Elixir's sigils:

https://codecodeship.com/blog/2024-06-03-curl_req

# orderedItems[].object.contentMap.en
Long form announcement of CurlReq, which became a kind of love letter to Elixir's sigils:

https://codecodeship.com/blog/2024-06-03-curl_req

# orderedItems[].object.content
For the non-Elixir people, a sigil is a language feature of Elixir (sugar) that allows easy access to macros. This makes it an attractive target for translating literals into Elixir data structures.

Examples:

UTC DateTime Sigil:

~U(2024-06-03T14:43:00Z)

Regex sigil:

~r/[0-9]*/

And now, curl sigil (translates to %Req.Request{}):

~CURL(curl https://catfact.ninja/fact)

--

NB You could also write

"curl https://catfact.ninja/fact" |> CurlReq.from_curl()

But this doesn't have the same impact imo.

# orderedItems[].object.contentMap.en
For the non-Elixir people, a sigil is a language feature of Elixir (sugar) that allows easy access to macros. This makes it an attractive target for translating literals into Elixir data structures.

Examples:

UTC DateTime Sigil:

~U(2024-06-03T14:43:00Z)

Regex sigil:

~r/[0-9]*/

And now, curl sigil (translates to %Req.Request{}):

~CURL(curl https://catfact.ninja/fact)

--

NB You could also write

"curl https://catfact.ninja/fact" |> CurlReq.from_curl()

But this doesn't have the same impact imo.

# orderedItems[].object.content
🤔 We need more sigils in Elixir

So I built one!

📢📢 Announcing curl_req!

It implements the ~CURL sigil!

🎁 Available on Hex as curl_req

This lib provides 2 functions and a macro:
- to_curl/1
- inspect/2
- sigil_CURL/2

So you can go from req <=> curl and back.

⏬ Download it and give it a try! Star on GitHub! Submit a PR if you find a missing feature that you need!

💗/🔁 so others can find this too.

# orderedItems[].object.contentMap.en
🤔 We need more sigils in Elixir

So I built one!

📢📢 Announcing curl_req!

It implements the ~CURL sigil!

🎁 Available on Hex as curl_req

This lib provides 2 functions and a macro:
- to_curl/1
- inspect/2
- sigil_CURL/2

So you can go from req <=> curl and back.

⏬ Download it and give it a try! Star on GitHub! Submit a PR if you find a missing feature that you need!

💗/🔁 so others can find this too.

# orderedItems[].object.content
@calmeilles I heard someone say this once:

FOMO is a powerful drug

# orderedItems[].object.contentMap.en
@calmeilles I heard someone say this once:

FOMO is a powerful drug

# orderedItems[].object.content
🎊 The problem has been fixed in HighlanderPG 1.0.3!

Now HighlanderPG will poll for the lock, using SELECT pg_try_advisory_lock(x, y); which always returns immediately.

Benefit from this battle testing, get the library at the link below!

https://hex.codecodeship.com/package/h ighlander_pg

🔚/🧵

# orderedItems[].object.contentMap.en
🎊 The problem has been fixed in HighlanderPG 1.0.3!

Now HighlanderPG will poll for the lock, using SELECT pg_try_advisory_lock(x, y); which always returns immediately.

Benefit from this battle testing, get the library at the link below!

https://hex.codecodeship.com/package/h ighlander_pg

🔚/🧵

# orderedItems[].object.content
And so by running SELECT pg_advisory_lock(x, y); and simply waiting to get the lock, I was causing a transaction to run indefinitely. We all know the BEAM's legendary reliability; this transaction could be waiting for WEEKS if you aren't regularly deploying.

6/🧵

# orderedItems[].object.contentMap.en
And so by running SELECT pg_advisory_lock(x, y); and simply waiting to get the lock, I was causing a transaction to run indefinitely. We all know the BEAM's legendary reliability; this transaction could be waiting for WEEKS if you aren't regularly deploying.

6/🧵

# orderedItems[].object.content
So back to HighlanderPG. How does a single SELECT query cause all this to happen? What I neglected to remember (it happens) is that EVERY statement in Postgres gets executed in a transaction. If you don't run BEGIN... END, Postgres does it for you.

5/🧵

# orderedItems[].object.contentMap.en
So back to HighlanderPG. How does a single SELECT query cause all this to happen? What I neglected to remember (it happens) is that EVERY statement in Postgres gets executed in a transaction. If you don't run BEGIN... END, Postgres does it for you.

5/🧵

# orderedItems[].object.content
Because a long-running transaction then causes dead rows to accumulate! And THIS is a problem because when you do a SELECT query, Postgres has to read all the rows (also dead ones) to know which rows you need. Indexes can't save you since they also point to dead rows.

Now your database has to do way more work to process the same queries. And this trend continues until the transaction ends, meaning that a crash is inevitable if the transaction continues indefinitely.

4/🧵

# orderedItems[].object.contentMap.en
Because a long-running transaction then causes dead rows to accumulate! And THIS is a problem because when you do a SELECT query, Postgres has to read all the rows (also dead ones) to know which rows you need. Indexes can't save you since they also point to dead rows.

Now your database has to do way more work to process the same queries. And this trend continues until the transaction ends, meaning that a crash is inevitable if the transaction continues indefinitely.

4/🧵

# orderedItems[].object.content
What are dead rows anyways? Postgres runs on MVCC - Multi Version Concurrency Control. Basically, an UPDATE does not change a row in-place.

Instead, a new row is written, and the old row becomes "dead". Dead rows can't be deleted right away, because they might still be visible to another transaction!

So VACUUM comes along later and cleans them up. But, VACUUM is smart, it doesn't delete any dead rows that a running transaction might be able to see! This is a problem, ...

3/🧵

# orderedItems[].object.contentMap.en
What are dead rows anyways? Postgres runs on MVCC - Multi Version Concurrency Control. Basically, an UPDATE does not change a row in-place.

Instead, a new row is written, and the old row becomes "dead". Dead rows can't be deleted right away, because they might still be visible to another transaction!

So VACUUM comes along later and cleans them up. But, VACUUM is smart, it doesn't delete any dead rows that a running transaction might be able to see! This is a problem, ...

3/🧵

# orderedItems[].object.content
HighlanderPG runs this query, and simply waits to get the lock. When it has the lock, it starts your child process. When the child process shuts down or dies, the lock is released and the cycle repeats, with another instance of HighlanderPG getting the lock.

HighlanderPG only runs this query, no "BEGIN", nothing like that. And simply holding an advisory lock (on session level) is not sufficient to cause dead rows to start accumulating. I was at a loss. But I was able to reproduce.

2/🧵

# orderedItems[].object.contentMap.en
HighlanderPG runs this query, and simply waits to get the lock. When it has the lock, it starts your child process. When the child process shuts down or dies, the lock is released and the cycle repeats, with another instance of HighlanderPG getting the lock.

HighlanderPG only runs this query, no "BEGIN", nothing like that. And simply holding an advisory lock (on session level) is not sufficient to cause dead rows to start accumulating. I was at a loss. But I was able to reproduce.

2/🧵

# orderedItems[].object.content
💥PG advisory locks considered dangerous. The following query can crash your DB.💥

HighlanderPG uses these locks to ensure that your process (data syncer, email sender, cron job runner, etc) is running on just one of your nodes.

Last week a customer called to say that their DB was getting crushed under the load when running HighlanderPG. The # of dead rows was getting very high, indicating some issue with VACUUM.

I quickly realized that something was causing a long-running transaction.

1/🧵

# orderedItems[].object.contentMap.en
💥PG advisory locks considered dangerous. The following query can crash your DB.💥

HighlanderPG uses these locks to ensure that your process (data syncer, email sender, cron job runner, etc) is running on just one of your nodes.

Last week a customer called to say that their DB was getting crushed under the load when running HighlanderPG. The # of dead rows was getting very high, indicating some issue with VACUUM.

I quickly realized that something was causing a long-running transaction.

1/🧵

# orderedItems[].object.content
🤓 Elixir Pro Tip

If the latest version of a package is broken, and you want to pin it to a version that works, it is nice to specify that you do want the next version after that.

Saves wondering later why that version is pinned.

#elixir

# orderedItems[].object.contentMap.en
🤓 Elixir Pro Tip

If the latest version of a package is broken, and you want to pin it to a version that works, it is nice to specify that you do want the next version after that.

Saves wondering later why that version is pinned.

#elixir

# orderedItems[].object.content
🚨 I released a new library for #Elixir today, called HighlanderPG.

It's a sequel to the Highlander lib I released 4 years ago, which lets you make a single process into a singleton in your cluster (so it only runs on one node). HighlanderPG solves a bunch of problems with the original.

I am doing something new with this library -- I am putting it up for sale. It's not a huge library, so the price is also quite reasonable -- $100 per year.

Read about it here: https://codecodeship.com/blog/2024-04-11-introducing-highlander-pg

# orderedItems[].object.contentMap.en
🚨 I released a new library for #Elixir today, called HighlanderPG.

It's a sequel to the Highlander lib I released 4 years ago, which lets you make a single process into a singleton in your cluster (so it only runs on one node). HighlanderPG solves a bunch of problems with the original.

I am doing something new with this library -- I am putting it up for sale. It's not a huge library, so the price is also quite reasonable -- $100 per year.

Read about it here: https://codecodeship.com/blog/2024-04-11-introducing-highlander-pg

# orderedItems[].object.content
@shadowfacts Previously they were hiring for Elixir (points to Pleroma) but now their only posting asks for Ruby (points to Mastodon). https://careers.tmtgcorp.com/p/c0f962f eea30-open-source-engineer

# orderedItems[].object.contentMap.en
@shadowfacts Previously they were hiring for Elixir (points to Pleroma) but now their only posting asks for Ruby (points to Mastodon). https://careers.tmtgcorp.com/p/c0f962f eea30-open-source-engineer

# orderedItems[].object.content
€8 logic analyzer from china came in, time to reverse engineer my ventilation system!

# orderedItems[].object.contentMap.en
€8 logic analyzer from china came in, time to reverse engineer my ventilation system!

# orderedItems[].object.content
@danluu I am once again very happy I went with Phoenix Liveview for my project. ~50kb compressed for a fully dynamic experience is pretty good.

No truly free cake though, you have to give up having a first-class offline experience. I don't get the idea that this is a concern for 99% of JS webapps though.

(I have also moved everything I could from JS to running on the server, including markdown parsing and syntax highlighting using Rust FFI)

# orderedItems[].object.contentMap.en
@danluu I am once again very happy I went with Phoenix Liveview for my project. ~50kb compressed for a fully dynamic experience is pretty good.

No truly free cake though, you have to give up having a first-class offline experience. I don't get the idea that this is a concern for 99% of JS webapps though.

(I have also moved everything I could from JS to running on the server, including markdown parsing and syntax highlighting using Rust FFI)

# orderedItems[].object.content
I love building front-ends on top of Oban. Oban's jobs table is easy to query, it has indexes in all the right places. And if you sprinkle in a little bit of Phoenix.PubSub in there, you can make a very nice UI in LiveView for your users showing the progress of batch jobs for instance.

I think it is a less-explored but really powerful way to use Oban.

#myelixirstatus

# orderedItems[].object.contentMap.en
I love building front-ends on top of Oban. Oban's jobs table is easy to query, it has indexes in all the right places. And if you sprinkle in a little bit of Phoenix.PubSub in there, you can make a very nice UI in LiveView for your users showing the progress of batch jobs for instance.

I think it is a less-explored but really powerful way to use Oban.

#myelixirstatus

# orderedItems[].object.content
Heading out to Brussels this evening to attend #FOSDEM tomorrow! Who else?

# orderedItems[].object.contentMap.en
Heading out to Brussels this evening to attend #FOSDEM tomorrow! Who else?