Loading...

Follow urmaul.com on Feedspot

Continue with Google
Continue with Facebook
or

Valid

I'm currently learning functional programming with scala and as a practice I've implemented FizzBuzz. To be honest, I made three different implementations of FizzBuzz but only the third one is good.

Stream.from(1) # Create infinite lazy stream from 1
  .map { (_, "") } # convert it to (number, word) tuple
  .map { x => if (x._1 % 3 == 0) (x._1, x._2 + "Fizz") else x } # Add "Fizz" to each 3rd word
  .map { x => if (x._1 % 5 == 0) (x._1, x._2 + "Buzz") else x } # Add "Buzz" to each 5th word
  .map { x => if (x._2 != "") x._2 else x._1.toString } # Take word or number
  .take(30) # Limit stream length
  .foreach(println) # Run everything and print results

It's even much better than usual imperative implementation and here's why.

Because composition. It's implemented as combination of small functions. This makes it easier to combine logic in different ways. Want to add one more word to be shown on every 7th step? Easy, just copy one line. Want to make it configurable? A bit harder, but still obvious how to do it.

Because proper logic grouping. Business logic is separated from interface work. First we make an infinite lazy steam with all the fizzbuzzing logic hidden inside. Then we limit it. Then we run everything and print results. That's very different from classic imperative approach where we are doing everything in a single loop. How do you cover the logic of imperative FizzBuzz with init tests? Probably, by hiding println inside dependency injected class. How do you cover functional FizzBuzz? Make a test for everything except last line. And there's no memory overhead because you never store whole stream in memory and only first element is calculated before you run foreach.

Functional programming teaches you to wrap all the logic into single pure function and decide what to do with it later. Isn't it cool?

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

React Styleguidist uses external markdown files to store usage examples. We wanted to use typescript for examples because reasons and we managed to do this with horrible solution. It includes custom webpack loader that parses typescript file with regular expressions and converts it to markdown. You could find parts of our code below.

AlertBar.example.tsx contains usage example. styleguide.config.js contains parts of config replacing Styleguidist's webpack loader with examples-loader.js. examples.test.tsx is a usual Jest test covering all examples with snapshot tests.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Imagine you have two groups of date ranges and you want to determine whether they overlap.

Nah, let's make it harder. Imagine you have two groups of include date ranges and two groups of exclude date ranges. Your task is to determine, whether there's a date that is present in both include groups and not present in exclude groups. How to do that?

Gotta warn you that in my case it was a config validation. Which means it's performed seldom and range groups are quite small. In this case algorithm simplicity was much more important than speed.

The algorithm works in 2 stages: convert ranges to array of events and follow these events to update the state.

Stage 1: Convert ranges to array of events

We need an array of events grouped by date. Each event contains information like "include group 1 range starts" or "exclude group 2 range ends".

function addGroupToEvents(array $events, array $ranges, string $key): array
{
    foreach ($ranges as $range) {
        $startDate = $range['start'];
        $endDate = $range['end'];

        // Add defaults
        $events += [
            $startDate => ['starts' => [], 'ends' => []],
            $endDate => ['starts' => [], 'ends' => []],
        ];

        // Add events
        $events[$startDate]['starts'][] = $key;
        $events[$endDate]['ends'][] = $key;
    }

    return $events;
}

$events = [];
addGroupToEvents($events, $includeGroup1, 'include1');
addGroupToEvents($events, $includeGroup2, 'include2');
addGroupToEvents($events, $excludeGroup1, 'exclude1');
addGroupToEvents($events, $excludeGroup2, 'exclude2');
ksort($events);

/*
$events = [
    '2018-01-01' => [
        'starts' => ['include1'],
        'ends' => [],
    ],
    '2018-01-10' => [
        'starts' => ['include2'],
        'ends' => ['include1'],
    ],
    ...
];
*/

Date format 'yyyy-mm-dd' allows you to sort dates as strings — it will just work.

Stage 2: Follow events to update the state

Now we track state of each range group and update it date by date. After each step we can check if state meets expected condition.

$state = [
    'include1' => 0,
    'include2' => 0,
    'exclude1' => 0,
    'exclude2' => 0,
];
foreach ($events as $date => $eventGroup) {
    foreach ($eventGroup['starts'] as $key) {
        $state[$key]++;
    }

    $isOverlapping =
        $state['include1'] > 0 &&
        $state['include2'] > 0 &&
        $state['exclude1'] == 0 &&
        $state['exclude2'] == 0;
    if ($isOverlapping) {
        // $date is the first date of ranges overlapping
        return true;
    }

    foreach ($eventGroup['ends'] as $key) {
        $state[$key]--;
    }
}

return false; // No overlap

Two things to notice:

  • We're processing "ends" after checking state. The easiest way to understand why is to imagine a one-day range.
  • We're storing state in integer instead of booleans for case of overlapping in same range group.
Summary

It's the simplest way I could solve this task. If you know better one — describe it to me in comments.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 
urmaul.com by Bogdan Kolbik - 5M ago

Let's imagine you have a script like below and you want to make it faster.

#!/bin/bash

echo "Started at `date +%Y-%m-%d\ %H:%M:%S`"

echo "Starting job 1"
sleep 5

echo "Starting job 2"
sleep 5

echo "Finished at `date +%Y-%m-%d\ %H:%M:%S`"

If it's ok to run jobs in parallel, you can easily do it with bash background jobs.

Simple case

By adding and ampersand sign (&) in the end of command you tell bash to run this command in background. Then you can wait command to wait until child background jobs finish.

#!/bin/bash

echo "Started at `date +%Y-%m-%d\ %H:%M:%S`"

echo "Starting job 1"
sleep 5 &

echo "Starting job 2"
sleep 5 &

wait

echo "Finished at `date +%Y-%m-%d\ %H:%M:%S`"
Multiple commands in background

That worked fine with single command, but can you run several sequential commands in one subprocess? Yes, by grouping them with { ... }.

#!/bin/bash

echo "Started at `date +%Y-%m-%d\ %H:%M:%S`"

{
    echo "Starting job 1"
    sleep 5
    echo "Finished job 1"
} &

{
    echo "Starting job 2"
    sleep 5
    echo "Finished job 2"
} &

wait

echo "Finished at `date +%Y-%m-%d\ %H:%M:%S`"
Limiting number of background jobs

Now let's imagine these commands are heavy and running them all will cause big load on your server. In that case it's possible to to limit background processes to a certain number.

The best way I found to do this is just to wait until there are less than JOBS_LIMIT jobs before running next one. We use jobs -rp command to list running child processes and to see their count.

#!/bin/bash

JOBS_LIMIT=5

echo "Started at `date +%Y-%m-%d\ %H:%M:%S`"

jobs=(1 2 3 4 5 6 7 8 9 10)
for job in "${jobs[@]}"
do
    while [ `jobs -rp | wc -l` -ge $JOBS_LIMIT ]
    do
        sleep 1
    done

    {
        echo "Starting job $job"
        sleep $(( ( RANDOM % 5 )  + 1 ))
        echo "Finished job $job"
    } &
done

wait

echo "Finished at `date +%Y-%m-%d\ %H:%M:%S`"
Conclusion

These tricks give you power to parallelize your scripts and make them much faster. If you want more tricks when running commands manually - check this article. If you have better ideas about running commands from bash scripts - don't hesitate to enlighten me in cooments.

And here's a bit of Nicholas Cage for you: .

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Imagine a wooden ship. It's a quite famous ship - Theseus himself used it to return to Athens from Crete. After that Athenians tried to preserve it by replacing decayed planks with new ones. Many years passed and now we can say for sure that every part of the ship was replaced at least once. Is it still the same ship?

The ship of Theseus is a well-known philosophical problem about meaning of identity. In the same time it's the good illustration of some programming concepts. To see that, let's look at original question ("Is it the same ship?") and two most popular answers for it: "no" and "yes".

It's not the same ship

Gottfried Wilhelm Leibniz thought objects are same if they share save set of properties. Which means the ship is not the same as soon an first part is replaced. Which also means "you cannot step into the same river twice". Which also means the ship of Theseus is a value object.

  • Identity of ship of Theseus is defined by set of planks it's made of.
  • After any plank is replaced it's another ship now.
  • Ship should be immutable so you can't call another ship with old reference "the ship of Theseus" by mistake.
  • Two ships built with same planks are equal and can be considered as same ship.

Well, last one sounds strange with real world objects. But still you can replace the word "ship" with "value object" and these rules remain true.

It's the same ship

Another point of view is that:

  • Ship remains same no matter how many planks you replace.
  • Two ships are different even they look same.

This leans to an assumption that ship has something that makes it the same over changes. Philosophers argue what that "something" means but developers don't have such problem. Usually it's ID of an entity the ship is.

Being a philosopher is hard. They use abstractions without clear definition (like the word "same") and then argue about what these abstractions mean. I wish developers didn't make same mistake so often.

And if you want to hear more about identity, check this lesson from awesome Crash Course Philosophy. It's about Batman.

Batman & Identity: Crash Course Philosophy #18 - YouTube
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 
urmaul.com by Bogdan Kolbik - 5M ago

I had to fix performance issues of one API endpoint. A pretty Symfony endpoint that gathers some data from database, assembling it to some structure and returns it as json.

Performance started being an issue when major part of that "some data" started to be 60000 entities. In worst case response time was almost 20 seconds. "Ok", I thought, "60k is a big enough number to make it slow". But trace showed that retrieving data from DB isn't a slowest part. There were things taking almost 1/3 of request time each. And these things were easy to fix.

N+1 requests are slow

We need to load and use related entities so we have a code like this:

$qb = $this->createQueryBuilder('a');

$qb
    ->select('a')
    ->innerJoin('a.entityB', 'b')
    ->where('a.foo = :foo')
    ->setParameter('foo', $foo);

$entities = $qb->getQuery()->getResult();

foreach ($entities as $entity) {
    $cId = $entity->getEntityB()->getEntityC()->getId();
    //...
}

It looks like we're joining the second table to make eager loading of related entity but we still make N+1 requests. That's because Doctrine doesn't create instances of related entity until we explicitly add that table to "select".

$qb = $this->createQueryBuilder('a');

$qb
    ->select('a, b')
    ->innerJoin('a.entityB', 'b')
    ->where('a.foo = :foo')
    ->setParameter('foo', $foo);

$entities = $qb->getQuery()->getResult();

//...

Now it works fine. Three chars removed one third of processing time.

Notice we're using EntityC in code but not joining this table. That's because actually we use Doctrine proxy entity. It stores EntityC id (from EntityB record) and makes DB query only when you try to get any another field.

Creating DateTime from string is slow

Now check this query:

$qb = $this->em->createQueryBuilder();

$qb
    ->select('f')
    ->from(Foo::class, 'f')

return $qb->getQuery()->getArrayResult();

Looks good. Hydrating 60000 objects looks slow so we're retrieving everything as arrays. Now it should be fast. Should be.

There's one nuance. Doctrine still hydrates every field you retrieve. That's pretty straightforward with all those strings and integers you have, but there are dates. To make every DateTime instance php has to run a function that tries to determine date format by looking at string and then to convert a string to internal representation of date and to time. That's not as slow as I'm trying to picture it. But making it 120000 times is slow. Making it 120000 and not using those objects is slow and sad.

So what's the deal? Fields "created_at" and "updated_at". Those ones are almost never used in code.

We're retrieving all fields of EntityA because we use almost all those fields. But still we should explicitly list fields we're gonna use.

$qb = $this->em->createQueryBuilder();

$qb
    ->select('f.bar, f.baz, f.spam, f.ham')
    ->from(Foo::class, 'f')

return $qb->getQuery()->getArrayResult();

Look at this query. We've removed half of processing time by updating one line. Easy!

Jms/Serializer is slow

Third one is hard to notice. We're using FOSRestController and just returning our data letting it to make response by itself.

Usually action looks like this:

use FOS\RestBundle\Controller\Annotations\View;

/**
 * @View()
 */
public function fooAction()
{
    //...
    return $result;
}

Under the hood FOSRestController uses Jms/Serializer - a pretty smart library for serializing stuff.

The problem is Jms/Serializer is smart even when it shouldn't. If you give it a huge multidimensional array - usually you just want it converted to json as is. But Jms/Serializer recursively walks along your array determining how to serialize every item one by one. With huge array that takes time. And that work is often useless because all you need is json_encode.

Let's update our controller a bit.

use Symfony\Component\HttpFoundation\JsonResponse;

public function fooAction()
{
    //...
    return new JsonResponse($result);
}

Now our endpoint takes much less than a second to work.

Notice, we didn't change any logic nor make any complex optimizations. All we did is removing unexpected time wasters. Now look into trace of your endpoint. And write in comments of you find something interesting there.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Imagine you want to rename form parameter but you also want it to accept old parameter name in requests to preserve backwards compatibility. Here's how you do it.

Step 1. Make tests for both cases

Symfony has a nice helper for testing forms. You should read about it in documentation.

Let's assume you want to accept parameter foo as parameter baz. You want both cases to be covered. Also you want predictable behavior in case both parameters are provided. Tests for these cases will look like this:

use Symfony\Component\Form\Test\TypeTestCase;

/**
 * @see MyType
 */
class MyTypeTest extends TypeTestCase
{
    public function testOldParameterName()
    {
        $model = new FooModel();
        $form = $this->factory->create(MyType::class, $model);

        $form->submit(['foo' => 'baz']);

        $this->assertEquals(
            'baz',
            $model->getBar(),
            '"foo" parameter must be accepted as "bar"'
        );
    }

    public function testNewParameterName()
    {
        $model = new FooModel();
        $form = $this->factory->create(MyType::class, $model);

        $form->submit(['bar' => 'baz']);

        $this->assertEquals(
            'baz',
            $model->getBar(),
            '"bar" parameter must be accepted'
        );
    }

    public function testBothParameterNames()
    {
        $model = new FooModel();
        $form = $this->factory->create(MyType::class, $model);

        $form->submit([
            'foo' => 'baz',
            'bar' => 'qux',
        ]);

        $this->assertEquals(
            'qux',
            $model->getBar(),
            '"bar" parameter must have priority over "foo"'
        );
    }
}

Run these tests. See them fail. Congratulations, you've done TDD. Now it's time to do work.

Step 2. Make aliases

There's no out-of-the box alias support in Symfony. If you check it's code, you'll see request parameters are tightly coupled with form field names. That's ok.

We can make an event listener that modifies request before passing to form. There we can convert old-style requests to new-style requests. To do this, add a block like this to your form code:

public function buildForm(FormBuilderInterface $builder, array $options)
{
    // Convert old parameter names to new
    $builder->addEventListener(FormEvents::PRE_SUBMIT, function(FormEvent $event) {
        $data = $event->getData();

        if (isset($data['foo']) && !isset($data['bar'])) {
            $data['bar'] = $data['foo'];
            unset($data['foo']);

            $event->setData($data);
        }
    });

    // ...
}
Step 3. Run tests again

See them green. Tadaa!

Nice job reading this, here's Nicholas Cage for you: . Now write something as comment.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Junior dev's life is hard an full of dangers. You come to technical interview and think "at least I'll learn something new from guys that know it better than me". Well, not really. Threre are some things that are ok in interviews and are totally wrong in real life. Here are some examples.

PHP Question: What would be the output of this code below?

$x = 5;
echo $x+++$x++;

Expected answer: 11

Real life answer: Just run it - that's not an rm -rf /. And then run git blame. I need to talk to person who wrote this. You shouldn't make unreadable code unless you have a real good reason to do it. And if you have that reason — first describe it in comment block so I know that you're not totally wrong. Something like /* My cat forced me to do that. Please call 911 */ would be ok.

JS Question: Please make a function isInteger.

isInteger(x)

Expected answer: [some code]... Oh, I forgot about numbers like 1e+21... [some another code]... Oh, I forgot about something Infinity... [cries silently]

Real life answer: This can be googled in 15 seconds. That's one of the most basic tasks — surely it was already solved. Why are you reinventing bicycle if someone smarter that you made it years ago?

JS Question: Please make a function isPalindrome.

console.log(isPalindrome("level"));                   // logs 'true'
console.log(isPalindrome("levels"));                  // logs 'false'
console.log(isPalindrome("A car, a man, a maraca"));  // logs 'true'

Expected answer: Here, it works.

function isPalindrome(str) {/* some code */}

console.log(isPalindrome("level"));                   // logs 'true'
console.log(isPalindrome("levels"));                  // logs 'false'
console.log(isPalindrome("A car, a man, a maraca"));  // logs 'true'

Real life answer: Here, it works and kinda unit tested.

function isPalindrome(str) {/* some code */}

console.log(isPalindrome("level") === true);
console.log(isPalindrome("levels") === false);
console.log(isPalindrome("A car, a man, a maraca") === true);

Because without tests nobody knows that it works.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 
urmaul.com by Bogdan Kolbik - 5M ago

So you want to shrink images with php and imagemagick. Here's samples of all filters so you can select the one you like most.

Near every image there's an average time spent to make this picture. And there's an easy way to compare two neigbouring filters - just hit one "show/hide" checkbox several times in a row.

In case you want to make the same test with any image - here's a tiny app to resize any image to any width with every filter.

Show/Hide filters scaleImage FILTER_POINT FILTER_BOX FILTER_TRIANGLE FILTER_HERMITE FILTER_HANNING FILTER_HAMMING FILTER_BLACKMAN FILTER_GAUSSIAN FILTER_QUADRATIC FILTER_CUBIC FILTER_CATROM FILTER_MITCHELL FILTER_BESSEL FILTER_SINC FILTER_LANCZOS
scaleImage 8.8ms
FILTER_POINT 9.3ms
FILTER_BOX 8.6ms
FILTER_TRIANGLE 11.8ms
FILTER_HERMITE 12.2ms
FILTER_HANNING 39.9ms
FILTER_HAMMING 38.4ms
FILTER_BLACKMAN 31.6ms
FILTER_GAUSSIAN 27.3ms
FILTER_QUADRATIC 14.5ms
FILTER_CUBIC 26ms
FILTER_CATROM 19.7ms
FILTER_MITCHELL 22.4ms
FILTER_BESSEL 27.9ms
FILTER_SINC 29.2ms
FILTER_LANCZOS 26.4ms
scaleImage 9.2ms
FILTER_POINT 8.8ms
FILTER_BOX 6.3ms
FILTER_TRIANGLE 10.9ms
FILTER_HERMITE 11.4ms
FILTER_HANNING 25.3ms
FILTER_HAMMING 36.9ms
FILTER_BLACKMAN 35.6ms
FILTER_GAUSSIAN 36.6ms
FILTER_QUADRATIC 28.9ms
FILTER_CUBIC 28ms
FILTER_CATROM 34.2ms
FILTER_MITCHELL 29.6ms
FILTER_BESSEL 40ms
FILTER_SINC 39.5ms
FILTER_LANCZOS 29.6ms
scaleImage 12.5ms
FILTER_POINT 21.4ms
FILTER_BOX 23.6ms
FILTER_TRIANGLE 22.2ms
FILTER_HERMITE 32.1ms
FILTER_HANNING 43.3ms
FILTER_HAMMING 45.2ms
FILTER_BLACKMAN 41.8ms
FILTER_GAUSSIAN 30.2ms
FILTER_QUADRATIC 27.7ms
FILTER_CUBIC 32.1ms
FILTER_CATROM 16.3ms
FILTER_MITCHELL 14.7ms
FILTER_BESSEL 15ms
FILTER_SINC 16.4ms
FILTER_LANCZOS 10.8ms
scaleImage 8.8ms
FILTER_POINT 8.8ms
FILTER_BOX 4.8ms
FILTER_TRIANGLE 7.5ms
FILTER_HERMITE 11.4ms
FILTER_HANNING 16.6ms
FILTER_HAMMING 16.6ms
FILTER_BLACKMAN 12.8ms
FILTER_GAUSSIAN 10.3ms
FILTER_QUADRATIC 6.6ms
FILTER_CUBIC 6.9ms
FILTER_CATROM 8.4ms
FILTER_MITCHELL 14.2ms
FILTER_BESSEL 9.8ms
FILTER_SINC 12.6ms
FILTER_LANCZOS 10.4ms

Looks like FILTER_SINC creates best images.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 
urmaul.com by Bogdan Kolbik - 5M ago

No one asked me about this for months. And now it's done. Gravacage has it's own documented php library and site.

Do you really want to use that ugly monsters generated by gravatar as default avatars? No! They're not cool!

And you know what's cool. Nicolas Cage is cool.

So you should start using his photos as default avatars.

Now click this link and start being awesome: gravacage.urmaul.com.

Read for later

Articles marked as Favorite are saved for later viewing.
close
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Separate tags by commas
To access this feature, please upgrade your account.
Start your free month
Free Preview