Setup your Bitbucket replacement for your Mercurial repository

A long time ago the German DelphiPraxis community convinced me to abandon my previous strategy of saving my source code via zipping only and use a source code management system.

So I googled a little bit and found SVN.

After some days of getting used to it - I proudly announced: "I have switched to SVN"! And with that, the shit storm has begun.

Why aren't you using a distributed revision-control-system? Don't use this old SVN system! Uses Mercurial or Git!

So I took the first "Mercurial" and with a little help and a talk from MVP Uwe Raabe on one of the Delphi-Tage-Event,  I had to realize that this system "just" works very well and offers all the advantages I missed in SVN. ( Sorry Craig )

Pushing everything for backup to a network-driver or a USB-Stick works perfectly. So no need for the old SVN Server installation on my local server, because mercurial is file-based. But after some time I had to share my source code with other developers and here comes BitBucket into play.

The use of BitBucket was as easy as Mercurial itself. 

I've never used Git, but I was told that it was just like Mercurial. Unfortunately, this is not the case. Even the first time I used git, there were big differences and I immediately hated git. But this was not a big deal, because I could use Mercurial for my local repositories and use Git for sharing where necessary.

I know many more developers using Git that Mercurial, but I never got used to the remote and local heads for different users and the parallel branches are show differently comparing to Mercurial.

After Bitbucket broke away from Mercurial repositories and hg is no longer supported, I had to find an alternative.  After some testing, I was able to run the SCMManager on a virtual Ubuntu installation. With the online available tutorials, the installation was not complicated. Unfortunately, SSL did not work out of the box.

The SCMManager is a Java-Client and this is not so easy with the certbot for Let's Encrypt. As far as I know, you have to convert the certificates and install them into the java key store. I like the cronjob based renewal, so hand converting was not my goal.

I found a little hint, that the Nginx server could be used as a proxy server to do the https/SSL connection and leave the java installation without any changes.

Installing software on Linux with this apt-get stuff is really easy and I wish this kind of installation would also possible on Windows. One drawback often is the location or the necessary changes to config files by hand. In this case, I had to type the config file from scratch.

By adding a new A record to the nameserver I was able to run the certbot with the "--nginx" param script who not only installed the certificate but also added the necessary parameters to my handwritten config file. Well done - this kind of "service" is really nice...

With some tweaking and some reload/retries, the server was reachable over https!

Small repositories worked immediately, but large files could not be transferred and caused an hg error: "request entity too large". The reason for this problem could only be the installed proxy server because everything worked before. But also this problem could be solved by a little googling. 

So now I have my own Mercurial server. If you want to know more about this, please write this in the comments, then I will try to collect my google knowledge into another post.

How to write you own Serverside-Extention!

Serverside Extention (IIS or Apache)!

For many years I have my own webserver. In my case, this always was a Windows Server, because I want to do my serverside scripts in Delphi! Apart from that, I had no clue about a Unix/Linux Server!

Being part of the Apocalypse Coding Group opens this door and in many hours of development of our little game, I could prove, that the same ISAPI.dll could also be compiled and executed on a Ubuntu Apache Server! 

I love Delphi. 

Building a new website or web service in the past, I had to make a choice:

For the website:

1. ASP.NET (Too bad only up to D2007)
2. ISAPI.DLL

After using ISAPI.dll's for a long time I've more often used ASP.NET. The code-behind for a webpage is not locked by the server, so it was easy to update the DLL-files. This was also possible by using the well known "egg"-loader. Updating the server to a newer version suddenly prevented the loader to work and I had never the time to find the reason for this. That why I preferred ASP.NET and for the other ISAPI.dll's I always hat to start and stop my IIS-Server.

For the web service:

1. SOAP

SOAP was the way to go because it was so easy to use. Just define your Interface and use it as it is local in your application. The only drawback is the huge data that is transferred with every call. That's why I used this not in the "native" way but just to transport an encrypted, compressed binary stream.

This was the past... But as time goes by...

If your traffic is increasing and you want to do many more things with your server, there comes a time when you realize that a 2 core server with an ancient operating system is just not up to date anymore.

Virtualization is one way to go. In my case, I use Proxmox on a beefy 2 CPU, many cores,  many RAM Server with a RAID SSD drive... Nice...

This server handles two or more Windows 2019 Server and many Ubuntu Servers for small web- or microservices.

Of course, for web-services "we" are doing JSON/RESTfull web services these days. So what is needed to do all the fancy stuff? 

ISAPI.DLL's - best performance - native execution, no script interpretation. For a web-site there are two ways to go: (perhaps many more, but not for me)

1. Uses a "normal" HTML page and  AJAX background calls to get the dynamic content you need for your site.

2. Use the Webbroker implementation and deliver the content directly from your DLL. This content could also do REST-calls to get more data from the same or a different service.

Perhaps you also want some URL-Rewrite or another Server-side handling, in this case, you also need a Filter.dll.

This time I'll do it better...

With the newly installed Windows 2019-Server, I tried the simple Hello-World DLL that is included with the egg-loader. But the Server was unable to execute this little DLL. But a request to the URL just downloads the DLL. This was the same with other DLL's I just copied from the old server. (The Hello-World.DLL works without problems on the old server). After some googling. I found the "uses 32 bit - DLL's" setting. But with this setting, I got a 500 with a weird error message. Setting the error level to detailed brings more information. Spoiler: Do not google these errors! Everything you'll find is nonsense! Like: You have to disable compression or other things. Most of the ideas lead you to the Handler settings - also wrong. (of course, you have to install all the ISAPI IIS extensions, I assume you have done this). So what is the difference between the old and the new server? 

64-Bit!

OMG... Why is there a setting for 32-Bit DLL's if you have to use 64-Bit anyway? OK - Start 10.4.x, compile it to 64-Bit and try again. To bad the Loader-Version 2.0 is from 2005 and not Unicode. After some fixes - the same problems. Forgot to disable the 32-Bit setting. The Hello-Worlds.DLL brings no errors anymore but is download again with every call. The loader is "kind of " working. All log files have wide-strings. Unreadable. Again some Unicode-Fixes and recompiled the Hello-World.dll to 64-Bit...

And here we go... a nice "Hello" displayed in the browser. But the Egg-Loader is unable to update the *.run version with the *.update version. I think the IIS is still holding the DLL. (How is this possible). So I wrote an eMail to William Egge if he has any ideas... 

This error is impossible. The loader is working, the hello-world is working, a free library should also work and the IIS should have no knowledge about the site-loaded DLL. Why is the loader unable to rename the files?

OMG... Again... No rights! 

By giving the IIS-User the access rights everything is working perfectly! Of course, I wrote to William that everything is working now... Answer from Bill: "Ok Cool 😊"! Perhaps I will send him the changes or I will write a different version, that is also able to handle Filter.dll's. Not decided yet!

So here is the workflow:

1. After installing the Windows-Server and all necessary extensions.
2. Create a website!
3. Compile your WebBroker.DLL (64-Bit or matching the Server)
4. Create a directory outside of your web-page-path. (e.g. c:\API / c:\ISAPI / c:\Script)
5. Create a minimal web.config in this directory. (for debugging errorMode="Detailed")

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <handlers accessPolicy="Execute, Script" />
        <directoryBrowse enabled="false" />
        <httpErrors errorMode="Detailed" />
    </system.webServer>
</configuration>

6. Upload the DLL to this directory and rename it to WebBroker.run (or any other name).
7. Upload the loader.dll to the directory and rename it to WebBroker.dll (Same name as your DLL)
8. Add the IIS-User with access rights to this directory
9. Add an Application to your website in the IIS-Manager. Point to the ISAPI directory.

(The Name will be part of the URL) "API" -> https://www.yourdomain.com/API/WebBroker.DLL)

10. Read is not permitted and browsing is off, but anyway I would store log files outside of this directory.

With the redesign of the egg-loader, we are back in the game!

If you want to see a YouTube tutorial of this, please leave a comment. There will be a video about this as part of my Firemonkey Development Kit and the Server-Side of my JSON-Store implementation, but this is scheduled not before I've completed my #D.MVVM Framework series and more basic stuff of my FDK.

If you want a notification on new videos, please subscribe to my channel and hit the bell icon! Thank you, this helps to grow my channel!