Elixir for aspiring Erlang developers

For the “Independent Coursework” of the University of Applied Sciences Berlin I created the following presentation:

 

Target audience are students of the bachelors degree Computer Science of the University.

If anything is unclear (or god forbid, wrong) drop me a mail or a tweet

Powershell cmdlets with dynamic param AND $args don’t work

Over the weekend I tried to implement auto completion in Elixir’s  mix (in Windows). Unfortunately I didn’t make it without introducing some problems. So I didn’t committed my changes to upstream. Currently I try to reach some of the more renowned Elixir/Windows contributors, to discuss the changes .

Motivation

Under normal circumstances I don’t use more mix tasks then test, phoenix.server and release but sometimes you need this weird command, you just can’t remember. The command mix help  is your friend here as it shows you all available commands (project aware!). Yet I don’t like to look the documentation up, if I need just some information on spelling. For example in the beginning I often tried to start the phoenix project with mix phoenix.start (Hint: that does not work). I am used to auto completion in my development environments so I tried to extend mix  as well.

Background

As I am using the Powershell for all my command line related tasks and the default file extension of Powershell is ps1, my command mix  execute the mix.ps1  in the Elixir bin folder.

Approach

Powershell scripts can have auto completion of parameters with an so called [validateSet("Param1","Param2",...)] , which incorporate all valid parameters. Sadly this is of no help, if we have to hard code the possible values for the parameter. A possible solution to this problem is the usage of a DynamicParam with dynamic validateSet (good resource here). To test my various iterations I wrote down all test cases (sorry no automated testing yet).

Iteration 1

Changes

If you have a look at the original mix  script (here) you can see that the script locates the mix.bat , flattens possible array arguments (is this still needed?) and then execute the mix.bat  with the newly flattened arguments.

The first problem we see here is the usage of the $args array. As Keith Hill points out in this SO comment the $args array “… contain any argument that doesn’t map to a defined function parameter…”. Which introduces the first problem: The DynamicParam  ONLY works for defined function parameters.

I copied the linked resource (again, here) and moved the old script to the process block. Because we are creating a script and not a function the signature of function Test-DynamicValidateSet {...}  needs to be removed. To generate the validateSet I replaced the line $arrSet = ... with

This populates the $arrSet  with all valid task. I also changed the value of the variable $ParameterName  to 'Task'  and renamed the variable $Path  to $Task

Test

A short test shows, the command mix  does work, the command mix help  does not. Reason for that is, we assign the first value to the parameter $Task .

Iteration 2

Changes

The call to mix.bat in the last row now get the $Task parameter as well:

Test

mix  works, mixd help  works. Awesome! Lets try auto completion. mix [tab]  …

This is weird. The auto completion takes it times (this is actually the time mix help --names  takes to return all valid tasks) yet the auto completion fills in file names from that folder… To fix that we need to make it clear, that our dynamic parameter is actually the first parameter. So after we set the $ParameterAttribute.Position = 0  (it was 1) we repeat our test.

mix  works, mixd help  works, mix [tab]  works, mix he[tab]  works also. What about arguments to parameters? like mix help --names ?

Damn.

Iteration 3

Changes

OK, we need positional arguments. Lets add some.

I don’t like that approach, because this script will fail on having more than seven parameters (our dynamic and $p1  – $p6 ) with the aforementioned error message.

We also have to forward our new parameters to the mix.bat :

OK, besides the now unused “flatten possible array parameter” logic and our “it will fail on having eight or more parameters” problem, how good are we?

Test

All tests in the test cases pass. Yet we have some unfinished problems.

Problems with this solution

  1. We can have only a fixed amount of parameters. This is not a big problem (as we can add more parameters in the signature), but this is neither elegant nor good practice.
  2. We now completely omit the “flatten array logic”. I have to admit, I’m not sure if  this is still needed, so I asked the original contributor of this logic but still wait for response.
  3. Most of the code was copied from our resource. We clearly added some of our own logic, yet we probably shouldn’t use this code without asking for permissions. I asked the author if I could use this snippet and wait for a response.
  4. Even if I omit the “flatten array logic” I tripled the Lines Of Code. I don’t know if the auto complete feature is worth this much code (read about code as a liability here)

As soon as the problem 3 is clarified I will upload the file here. As soon as the other problems are clarified (and/or fixed) I create a pull request in GitHub to upstream the changes.

Elixir, Phoenix and Windows: Some insights on my 1000┬Ás problem

On Saturday I wrote about “Elixir, Phoenix and Windows: No faster responses than 1000 microseconds?“. I described two problems I had with Phoenix in Windows: My response times seem to be stuck at 1000 microseconds and Powershell couldn’t display the μ sign. I dug into some code and the mailing lists and found (with a lot of help) some answers.

1000 Microseconds on Windows?

The response times in Phoenix are often measured in Microseconds. Yet on Windows you won’t see any requests faster than 1000 Microseconds. That’s not because of a slower OS, but a not as precise as needed timer:

On Windows a developer has several options to get the system time. According to windowstimestamp.com there are (including the precision):

  • The time service (1 ms)
  • The performance counter (between 1 ms and 1 μs)
  • The GetSystemTimeAsFileTime API (100 ns)

The highest precision (100 nano seconds) can be achieved with the GetSystemTimeAsFileTime API (or the GetSystemTime API, which is the same data but differently formatted).

Actually this is the API  the Erlang VM (which provides the run time for Elixir and Phoenix) is using. So in theory we should be able to get more precise data out of these API. Yet the Erlang VM only returns millisecond as smallest unit. I’m pretty there are reasons for it, but I don’t understand them. If you are curious (and don’t fear a little bit C code) go ahead and look at the implementation of the timing in Erlang: sys_time.c in Erlang VM

1000┬Ás instead of 1000μs?

The second problem I had on saturday was the missing μ sign in my Powershell environment. I got hinted that I have to set the code page of Powershell to UTF8:

This fixed the problem for me. Unfortunately this introduced another, more grave bug to me: On putting out special characters in iex.bat, iex now fails to react completely. Until this bug is fixed, I strongly advise against this fix.

Elixir, Phoenix and Windows: No faster responses than 1000 microseconds?

If you read around phoenix developers you often hear stuff like “Awesome, requests served in under xxx microseconds!”. Yet if I try the Phoenix framework, I only have this results:

With special emphasis on [info] Sent 200 in 1000┬Ás . Here we have two problems:

  1. It looks like the command line doesn’t know the μ sign and replaces it with ┬Á.
  2. Did it really take 1000 microseconds to serve the request? I’m not sure, but I NEVER have a request faster than that. Sometimes I have a slower request (163 milliseconds on startup for example) but never faster than 1000 microseconds.

Locating the error source

Lets find the culprit: I use the Powershell to start the Phoenix app. Can Powershell show the μ sign?

Powershell using the μ sign
It can!
Powershell showing off and uses the the μ sign as variable name
Variables with the μ sign are possible as well

As a matter of fact, you can use the μ sign as variable, if you want to.

So Powershell is good. What’s the problem then? Looking into the mix.ps1 we can see that it executes the mix.bat, which executes the elixir.bat which either executes erl.exe or werl.exe. So lets have a quick look if the cmd (*.bat files are executed by cmd) is capable of the microsecond sign.

Also the command line is can show the sign
Also the command line can show the sign

So the problem isn’t the command line either. I printed the final executed call to erl.exe and executed this command without the cmd as middle man. The problem remained. So I assume it is a problem with either the erl.exe (if I use the –werl command line switch, werl.exe gets executed, the phoenix app gets started but no info is shown in werl.exe), Elixir, Phoenix or some of the plumbing in between.

Lets test Elixir. I created a new Elixir app mix new micro_second_test  and wrote a single function in it:

When starting the application with iex.bat -S mix  and executing the function we get this result:

So we can rule out Phoenix as Elixir already has that problem. What about erl.exe?

Erlang can show the μ sign. So the culprit is either Elixir or the plumbing. I will open an GitHub Issue for this problem. For the second problem (never showing less than 1000μs) I am not sure how to check. I think it could be in the cowboy web server, or in the Plug.Conn. But I have no clue…

Elixir, postgres and chocolatey

You try to use the phoenix getting started guide on windows and the task “mix ecto.create” fails with an useless error? Chances are your postgresql database isn’t available with the ecto default credentials “postgres”:”postgres”. Try to logon to the database with the credentials “postgres”:”Postgres1234″ and change the password for the user to “postgres”. Also, don’t forget to change the password of the Windows user “postgres”.

Update 15.11.2015: You also have to change the logon information for the service, otherwise postgresql won’t start after an restart.

Long version

If you happen to start with elixir and phoenix you will probably install postgres at some point. If you also happen to use Windows AND are a user of chocolatey (which you should be!) it could happen that you run in a nasty, not very helpful error message when you try to use the phoenix getting started guide on the mix task:

The error states exactly nothing:

Which is not that helpful. Because of the error message you can’t exactly google for that particular error. (I mean, yeah you could look the codepoint up but… really?)

Because the ecto tasks failed and ecto is the database mapper, I tried to connect to my recently installed postgres database. Which immediately made my mistake clear: ecto expects an user “postgres” with the password “postgres” for the database connection. But these credentials didn’t work!

I tried to find the default password for postgres (“postgres” being the only answer I found) but failed after a quick googling. So I uninstalled the postgres package via chocolatey

from my computer (which unfortunately didn’t work… I deleted the folder afterwards :-/ ) and reinstalled it.

On the installation log I found the default password afterwards:

I logged on to the database and changed the password:

Afterwards I changed the password for the windows user, “postgres” created on the installation accordingly.

Now the mix task works as expected.