Digitally Extrapolated Graphics via Algorithmic Strokes (DEGAS) – An Oil Painting Program

So for those of you following me on Facebook, I’ve lately been working on a programming project that takes photographs and turns them into impressionist paintings. I have named this program DEGAS, which stands for Digitally Extrapolated Graphics via Algorithmic Strokes. (I’ve always loved a good name-cronym, especially when it’s a pun. 🙂 )

It’s not done with neural networks, like Ostagram.ru does. Although neural networks are all the rage right now, they only mimic the features and product of a painting, not the process.

It’s also not a filter – that is to say that the process is not fully automated. You don’t just choose a photograph and it works on it alone and then *boom* you have a painting. You have to “paint” the details onto the photo by choosing your brushes, size, angle range, and scatter, and DEGAS automates color selection and brush orientation based upon what fits the color contours the best.

A before and after shot on a test image.

A before and after shot on a test image. All the pictures here are of my wife. 🙂

In essence, it’s an impressionist gattling-brush. The choice of what features to emphasize and their final form are still more or less in the hands of the painter, but the most time consuming aspects are automated.

It’s written in plain, vanilla HTML5/Canvas. No special libraries (although I tinkered with gpu.js for a few operations that didn’t pan out). And the currently used portion of the codebase comes in at under 1,000 lines. Once I work out a few bugs, and design a nicer UI for it, I’ll probably release the source code for anyone who wishes to play with it.

The Process

So, how does it work? First it creates several canvases:

  • The Display canvas.
  • A Source canvas to hold the initial image.
  • A Blur Buffer canvas which holds a copy of the initial image, but blurred to a certain value (more on that in a bit).
  • A general Buffer canvas for compositing things on the fly.
  • A Brush canvas to hold the brush’s shape; and
  • A Brush Buffer canvas where color matching takes place.

The source image is loaded in to the Source, Blur Buffer, and Display canvases. The Display canvas is appended to the body of the document, and the Blur canvas is blurred by a few pixels to smooth out rough details.

The initial brush shape is then loaded into the Brush canvas. All a brush is is an outline with a transparent background. Generally it also has antialiased edges and some alpha for thin paint. (Brushes are generally 512×512.)

Where there are several ways to add strokes to the painting, when each stroke is placed it follows the same procedure:

  1. First, it grabs the color of the pixel at the center of the stroke.
  2. It tints the Brush canvas to this color. All detail on the brush shape is lost, but all transparency is preserved.
  3. It then reads out what allowable angles it has to paint strokes. By default it has 16 equal angles between 0 and 180 degrees. I’ve had even better results with 20 angles all 360.
  4. For each allowable angle:
    1. It reaches into the Blur Buffer and takes the area of the image that the brush will be painted on and writes it to the Brush Buffer.
    2. It then paints the brush at the appropriate angle on top of the Brush Buffer.
    3. It then runs a comparison metric on the color difference between the original area on the Blur Buffer and the painted stroke over the same area on the Brush Buffer.
  5. It then paints the brush onto the Display Canvas in the orientation that fits the color and contour of the original image the best.

And it does this dozens of times per second to a very stunning effect.

Some of the parameters that can be changed to affect the spread of the strokes include:

  • Averaging the color of the stroke, or increasing the saturation, etc.
  • The blurriness of the Blur Buffer. This helps make better stroke choices when working with noisy images, or features like hair.
  • The shape of the brush – I have 9 shapes made from actual scanned brush strokes.
  • The size and scatter of the strokes under the mouse. One generally starts out with a large size and scatter (300,300) for large features, and then fills in the details with smaller strokes with less scatter (150,100 to 50,40).
  • What angles are allowable. The more angles, the slower the process as DEGAS needs to compare each iteration, but at the same time the better the match. Or if you want to force the brush in a certain direction you can do that, too.

DEGAS also keeps track of each stroke. As a result, it can re-paint an image while loading a painting back into memory; which looks like this:

And because it keeps track of all of those strokes, it can also do some really fun post-processing. It can generate a height map of the paint:

And from that height map, generate a normal map:

And then from that normal map, add the depth of the paint by simulating light casting onto the strokes:

And it looks pretty neat:

One could potentially also use the height map to “3D print” one of these paintings onto a canvas using something like a UV ink printer (which is precisely what they did for The Next Rembrandt project).

I’ll be posting more images here soon.

UPDATE (March 29, 2017): I’ve skipped the normal map step and have gone and implemented GIMPs bump map algorithm (which computes the normals much better and shades them directly without sticking them in a normal map file first). Here’s a new video showing it in action. You may have to view it full screen as the effect of the brush strokes at this resolution is a bit subtle.

FURTHER UPDATES (Oct 16): DEGAS is now officially an “award winning algorithm.” It came in 1st place in the Graphics category at Arts in the Park (I should post some pictures). I also have a gallery of some of my work up here.

Peace,
-Steve

Harriet Tubman On the $20 Bill

/home/content/04/10839404/html/steve/wp content/uploads/2016/04/160428 Harriet Tubman

This is the first $20 bill. It was issued in the 1860’s. Lady Liberty was on it.

We’re just bringing her back as Harriet Tubman.

Another important note: Andrew Jackson didn’t appear on the $20 bill until 1928.

Before that it was:

  • Grover Cleveland (1914).
  • George Washington (1905)
  • Hugh McCulloch (1902) (Not a President!)
  • John Marshall (1890) (Not a President!)
  • Daniel Manning (1886) (Not a President!)
  • James Garfield (1882)
  • Stephen Decatur (1878) (Not a President!)
  • Alexander Hamilton (1869) (Not a President!)
  • Pocahontas (1865) (Not a President!) and
  • An Eagle (1863) (Not a President…)

Lots of not-Presidents. 🙂

Peace,
-Steve

What is My Episcopal Identity?

This is my response to Episcopal Café’s #myepiscopalidentity campaign. 🙂

———-

/home/content/04/10839404/html/steve/wp content/uploads/2016/02/160208 Episcopal Shield

The Episcopal Shield

What makes me an Episcopalian? It’s in my DNA.

It’s a long tradition in my family carried by my mother, my grandmother, my great grandmother, and further back my family line. It’s a mantle that I now bear, and by example live and share with my children.

It is a Christian faith that sees the importance of Scripture, Tradition, and Reason – all as equals. It’s a faith that doesn’t make windows into other peoples’ souls. It’s a faith that strives to agree upon matters of importance, but allows a wide range of differing opinions and expressions with everything else. It is a faith that loves music. It’s a faith that recognizes the Creeds, the universal, catholic church, and the unbroken line of Bishops that trace all the way back to Jesus’ Disciples. And it is a faith that shows how the way we pray – together in Common Prayer – also shows the way that we believe. Lex orandi, lex credendi.

A number of ethnic churches tend to identify themselves by their origins (Greek Orthodox, Russian Orthodox, Polish Catholic, etc.) and keep the rich culture of their heritage as a vessel for how they express their devotion to Christ. With the Episcopal Church it is the same, but harder to see. Many do not realize that our tradition stretches all the way back to England where, in the mid 1500s, our ancestors began to share the rich liturgy in their native English for the very first time – during a period when virtually all of the language heard in church was Latin.

The Episcopal Flag

The Episcopal Flag

It was a tradition to be understood by the common person, and as such it has spread and touched upon everywhere the English language has traveled to – every continent – throughout what eventually became the modern Anglican Communion. To this day it leaves fingerprints in our greater culture and language, leaving idioms and phrases from rituals, from turns of phrase in Shakespeare to modern idioms such as, “read, mark, learn, and inwardly digest,” or “speak now or forever hold your peace.”

Sometimes, folks from other flavors of Christianity will ask me, “Is your church based on the Bible?” And I am one to point out that, “I could more say that it’s likely your Bible is based upon my church.” The King James Version that many denominations lean so heavily upon on was “Authorized” by the Church of England, and brought to America by what eventually became the Episcopal Church. The church that today claims among its own a full quarter of our country’s Presidents, great thinkers, scientists, and artists such as George Washington, Thomas Jefferson, Fred Astaire, Judy Garland, Nat King Cole, Franklin Delano Roosevelt, Charlton Heston, Madeline L’Engle, Buzz Aldrin, and Robin Williams, but to name a few.


Judy Garland & Fred Astaire in “Easter Parade” (1948)… It doesn’t get much more Episcopalian than that. 🙂

We are educated, because we see the value in education. We hold the most graduate and post graduate degrees per capita of any Christian denomination in the United States. We are not afraid to use them.

C Teesdale Anglican ExampleMagnificat Text

Anglican Chant

Where our liturgy has come a long, long way it is still based upon essentially the same patterns and formularies that have been used for over 500 years. It is a liturgy which is predominantly made up of reciting portions of Scripture, chanted Psalms, Canticles, and readings from the Lectionary.

The 8th SacramentWe have services with thees and thous – but in the spirit of keeping the liturgy understandable we have services modern English as well (our Rites I and II). We also have a rich Daily Office, founded upon Thomas Cranmer’s original abbreviation of the seven Divine Hours down to Morning and Evening Prayer, with the distinctly Episcopalian Lesser Hours of Noonday and Compline to compliment them. We love our coffee hour after the Eucharist.

We focus on community service – feeding the poor, clothing the needy, housing the homeless – and strive to see Christ in all people, no matter who they are. We are part of a greater whole, locally adapted, and are always out to do the right thing, no matter how tough.

There is more to be said. There is always more to be said. But all of this boils down to #myepiscopalidentity.


Steve Caruso, MLIS
Christ Church, New Brunswick, NJ
ChristChurchMini

2d6-2d6 as a Base Roll For Roleplaying Games

So I am one to sometimes meddle with making roleplaying systems (in fact I’ve been playing games in that system for years now)  but in the afterglow of storytelling at PrinceCon for the first time in a long time, I more seriously started to ponder about dice. The PrinceCon system — now with more than 40 years of history behind it — presently runs off of a d20 OGL-based system which (like all d20-based games) resolves randomness with a roll of a 20-sided die, plus modifiers, to meet or exceed a difficulty.

d20 probabilities to roll "at least" the number shown. From anydice.com.

d20 probabilities to roll “at least” the number shown. From anydice.com.

A single die roll, of course, is a completely flat distribution. You have a 5% chance to land on any one number and a 50% chance to roll an 11 or higher.

3d6-graph

The 3d6 bell curve. From anydice.com.

I’m not one who likes flat distributions (they’re all over the place) so when I play d20 I tend to play with the 3d6 variant, which makes a nice bell curve (seen above) whose peak falls betwixt 10 and 11, and whose extremes (3 and 18) have a mere ~0.5% chance of happening.

Probabilities for landing at least on the number listed for 3d6. From anydice.com.

Probabilities for landing at least on the number listed for 3d6. From anydice.com.

This works well for many things, but it’s not as elegant as a d20 in some respects. First, a d20 has a range of 20 values. In a system where a value of 10 is average (like d20 OGL) that works out well. 3d6 only has a range of 16 values (3-18), so it is much shorter and by default, difficulties below 3 are impossible (which effectively takes out a chunk of range). Furthermore, none of the values on the curve are nice “round numbers.” Getting a 10 (base) is a 62.5% chance, getting a 15 (+5 base) is ~5%, and getting an 18 (+8 base) is about 1 in 200.

Playing around with probabilities on AnyDice.com (awesome site) I fiddled around with different possible dice combinations and unexpectedly found that 2d6-2d6 might present a more elegant solution.

2d6-2d6 bell curve. From anydice.com.

2d6-2d6 bell curve. From anydice.com.

Where it seems like a lot of dice (and a bit more math) 2d6-2d6 creates a beautiful graph with some elegant properties.

  • First of all, it’s a nice bell curve.
  • Second, the peak of the bell curve is on 0, so all results will fall around the base modifier. In a system where the base statistic is 10 that means that a roll will range between 0 and 20 (just like d20, albeit 0 is possible).
  • Third the major increments fall upon rather round probabilities:
"At least" rolls on 2d6-2d6. From anydice.com.

“At least” rolls on 2d6-2d6. From anydice.com.

  • About 90% of the time you’ll at least hit -4 or more.
  • About 75% of the time you’ll hit -2 or more.
  • About 2/3rds of the time you’ll hit -1 or more.
  • Base probability (0) is 55% (as close to half as one can get)
  • +2 is almost exactly 1 in 3
  • +3 is roughly 25%
  • +5 is roughly 10%
  • +6 is roughly 5%
  • +7 is roughly half that (~2.5%)
  • +8 is roughly 1%
  • +9 is roughly 1 in 250; and for something truly epic
  • +10 is roughly 1 in 1000

This progression makes incremental changes to the difficulty of a task ease into a curve with some nice round numbers. 🙂

And given that the extremes happen so rarely, it gives a good excuse for an epic resolution as they really are a one-in-a-thousand shot.

all-of-them

A comparison of all three curves. From anydice.com with modifications.

 

Now, this has direct application to d20-like games. The only thing that would have to change is the calculation of base skill levels and roll modifiers as now everything would need to start out at a base of 10. In the end you would get the same range as a d20, but with a nice bell curve.

Mean Deviation: Range:
d20 10.5 5.7 20 [1-20]
3d6 10.5 2.96 16 [3-18]
2d6-2d6 0 3.42 21 [-10-10]

As such I believe that there is some good potential for using 2d6-2d6 as a base roll for table-top RPGs.

Peace,
-Steve

A Quandry: The “Earliest Fragment of Mark”

I have a question for you.

With this whole purported 1st century Mark fragment found in the papier-mâché egyptian funerary mask thing that’s hit the media, the biggest quandry in my mind is:

Assuming that such a retrieval would be done by trained professionals in a controlled environment without any pressures from fundamentalist corners, what is more valuable to the world: A funerary mask? Or the texts it is potentially made up of?

It’s not an easy question.

What do you think?

These are my thoughts:

Fake Gold Eagle vs real

A fake bullion coin and a real one. I’ll leave it to you to figure it out.

When looking at, for example, high quality counterfeits of expensive coins, I could be faced with a similar (albeit not identical) dilemma. There are presently a glut of fake American Gold Eagles on the market which are made of thickly gold-plated tungsten. Some of them have such carefully made strikes that the only way to truly confirm if they’re fake (when you have a “hunch” and have tested it via other means) is to destroy them, mainly drill into their center and see if the gold goes all the way through.

tungstenfakecoin

A destroyed edge revealing a tungsten core.

Where Gold Eagles are relatively common (they’re a bullion coin currently minted) drilling one would destroy its premium (a few hundred dollars over the gold it’s made out of) and most of its collectibility. In this case, the question becomes “Is it worth it?” when there’s suspicion that it’s fake.

Saint-Gaudens_high_relief_double_eagle

A picture of the 1907 “Ultra High Relief” St. Gaudens Double Eagle. A major rarity.

The question’s weighting changes when we’re no longer looking at a common but valuable coin, but instead consider a numismatic rarity of which there are only a small number of known examples.

Do we chance destroying it? Not as readily. It seems to be our duty to exercise every possible non-destructive test first and then, and only then, if there is still suspicion, we try other options.

The quandry truly is where that threshold lies, and with egyptian funerary masks, how its commonality or rarity influences the risk of disassembling it over the prospective gains to the corpus of ancient texts. If it’s one of a number of well-known examples that are well documented or if it’s a singularly unique example potentially have weight.

Some will say “There is no acceptable threshold. It’s a cultural relic and the mask is priceless.” where others will say “The texts it’s made up of represent dozens of cultural relics which combined are worth more to humanity than the intact mask.”

Sadly, there is no “right” answer so long as chance plays a part.

Peace,
-Steve

“You Can’t Sink This Ship” – Ken Ham

So Ken Ham is presently running ads for his ark park thing (despite the fact that he probably doesn’t have the money to do so — you can google it for the whole debacle).

They presently look like this:

Qqd5aGz

I believe that a similar thing was said about another ship, so an adjustment is in order:

Cant Sink This Ship

I shall await Ham’s pending hubris to resolve.

I do not expect that he’ll take the hint.

Peace,
-Steve