How to write you own Serverside-Extention!

Serverside Extention (IIS or Apache)!

For many years I have my own webserver. In my case, this always was a Windows Server, because I want to do my serverside scripts in Delphi! Apart from that, I had no clue about a Unix/Linux Server!

Being part of the Apocalypse Coding Group opens this door and in many hours of development of our little game, I could prove, that the same ISAPI.dll could also be compiled and executed on a Ubuntu Apache Server! 

I love Delphi. 

Building a new website or web service in the past, I had to make a choice:

For the website:

1. ASP.NET (Too bad only up to D2007)
2. ISAPI.DLL

After using ISAPI.dll's for a long time I've more often used ASP.NET. The code-behind for a webpage is not locked by the server, so it was easy to update the DLL-files. This was also possible by using the well known "egg"-loader. Updating the server to a newer version suddenly prevented the loader to work and I had never the time to find the reason for this. That why I preferred ASP.NET and for the other ISAPI.dll's I always hat to start and stop my IIS-Server.

For the web service:

1. SOAP

SOAP was the way to go because it was so easy to use. Just define your Interface and use it as it is local in your application. The only drawback is the huge data that is transferred with every call. That's why I used this not in the "native" way but just to transport an encrypted, compressed binary stream.

This was the past... But as time goes by...

If your traffic is increasing and you want to do many more things with your server, there comes a time when you realize that a 2 core server with an ancient operating system is just not up to date anymore.

Virtualization is one way to go. In my case, I use Proxmox on a beefy 2 CPU, many cores,  many RAM Server with a RAID SSD drive... Nice...

This server handles two or more Windows 2019 Server and many Ubuntu Servers for small web- or microservices.

Of course, for web-services "we" are doing JSON/RESTfull web services these days. So what is needed to do all the fancy stuff? 

ISAPI.DLL's - best performance - native execution, no script interpretation. For a web-site there are two ways to go: (perhaps many more, but not for me)

1. Uses a "normal" HTML page and  AJAX background calls to get the dynamic content you need for your site.

2. Use the Webbroker implementation and deliver the content directly from your DLL. This content could also do REST-calls to get more data from the same or a different service.

Perhaps you also want some URL-Rewrite or another Server-side handling, in this case, you also need a Filter.dll.

This time I'll do it better...

With the newly installed Windows 2019-Server, I tried the simple Hello-World DLL that is included with the egg-loader. But the Server was unable to execute this little DLL. But a request to the URL just downloads the DLL. This was the same with other DLL's I just copied from the old server. (The Hello-World.DLL works without problems on the old server). After some googling. I found the "uses 32 bit - DLL's" setting. But with this setting, I got a 500 with a weird error message. Setting the error level to detailed brings more information. Spoiler: Do not google these errors! Everything you'll find is nonsense! Like: You have to disable compression or other things. Most of the ideas lead you to the Handler settings - also wrong. (of course, you have to install all the ISAPI IIS extensions, I assume you have done this). So what is the difference between the old and the new server? 

64-Bit!

OMG... Why is there a setting for 32-Bit DLL's if you have to use 64-Bit anyway? OK - Start 10.4.x, compile it to 64-Bit and try again. To bad the Loader-Version 2.0 is from 2005 and not Unicode. After some fixes - the same problems. Forgot to disable the 32-Bit setting. The Hello-Worlds.DLL brings no errors anymore but is download again with every call. The loader is "kind of " working. All log files have wide-strings. Unreadable. Again some Unicode-Fixes and recompiled the Hello-World.dll to 64-Bit...

And here we go... a nice "Hello" displayed in the browser. But the Egg-Loader is unable to update the *.run version with the *.update version. I think the IIS is still holding the DLL. (How is this possible). So I wrote an eMail to William Egge if he has any ideas... 

This error is impossible. The loader is working, the hello-world is working, a free library should also work and the IIS should have no knowledge about the site-loaded DLL. Why is the loader unable to rename the files?

OMG... Again... No rights! 

By giving the IIS-User the access rights everything is working perfectly! Of course, I wrote to William that everything is working now... Answer from Bill: "Ok Cool 😊"! Perhaps I will send him the changes or I will write a different version, that is also able to handle Filter.dll's. Not decided yet!

So here is the workflow:

1. After installing the Windows-Server and all necessary extensions.
2. Create a website!
3. Compile your WebBroker.DLL (64-Bit or matching the Server)
4. Create a directory outside of your web-page-path. (e.g. c:\API / c:\ISAPI / c:\Script)
5. Create a minimal web.config in this directory. (for debugging errorMode="Detailed")

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <handlers accessPolicy="Execute, Script" />
        <directoryBrowse enabled="false" />
        <httpErrors errorMode="Detailed" />
    </system.webServer>
</configuration>

6. Upload the DLL to this directory and rename it to WebBroker.run (or any other name).
7. Upload the loader.dll to the directory and rename it to WebBroker.dll (Same name as your DLL)
8. Add the IIS-User with access rights to this directory
9. Add an Application to your website in the IIS-Manager. Point to the ISAPI directory.

(The Name will be part of the URL) "API" -> https://www.yourdomain.com/API/WebBroker.DLL)

10. Read is not permitted and browsing is off, but anyway I would store log files outside of this directory.

With the redesign of the egg-loader, we are back in the game!

If you want to see a YouTube tutorial of this, please leave a comment. There will be a video about this as part of my Firemonkey Development Kit and the Server-Side of my JSON-Store implementation, but this is scheduled not before I've completed my #D.MVVM Framework series and more basic stuff of my FDK.

If you want a notification on new videos, please subscribe to my channel and hit the bell icon! Thank you, this helps to grow my channel!

Workflow and multi - binding with #D.MVVM - The Delphi MVVM Framework!

Workflow and multi binding with #D.MVVM!

In every example for MVVM, you find the three blocks diagram explaining the binding between the View, the ViewModel, and the Model.

If you dig deeper into a real-world application, you may use the 1:1:1 relation, but in most cases, you need it very differently.

1:1:1 means One View connected to one ViewModel, connected to one Model!

Let's start with the View:

Most of my views are not simple TForms but much more assembled on small subframes. The subframes are also TForms and not TFrames, because of the recurring problems with frames (in the past). But that is not the topic.

The View will be created in the composition root - or elsewhere - and then?

Every subframe must also be created and placed into the target frame of the parent view.

So far so good. Of course, this is all handled by my framework.

In some cases, every frame should have its own ViewModel. But what if not?

Well, keep this in mind and start at the other end of this chain...

We have a database where every row is just an entry of our beloved TPerson, with the normal address fields.

Part of this TPerson is a list of telephone numbers handled in a different table and perhaps also a list of bank accounts also in a different table.

As we all know - I hope - the Model is not our Database interface, the Model will get the interface to the database for local or remote storage. This could be an interface to a local database or to a RESTfull web service or something else!

Let us talk concretely for the moment:

To store a value into a database or send it as JSON you need the field-value. For the complete row and perhaps for the subtables you need a list of field-values and the field-names or at least the index position in the param list of the query.

We will name this: Place X.

Place X need the Name, Street, Zip and more... And of course, the two lists for the subtables.

To be able to get this right you have to collect the data. A good place would be the Model. Wait... The Model?

So for all the different fields and from at least 3 frames, we have to collect the data in only one Model!

To store the data, the database should do a transaction to keep the subtables in sync with the address table. This would be a nightmare if 3 Models would do this separately.

At this point, we have our View with 3 subframes (Address, Telefonlist, Bankaccountlist) and only one Model.

1(3) : ? : 1

The View is not the place to collect the data from the subframes. Should we bind the subframes to one ViewModel?

In this case, we get our example relation of 1:1:1. That would be nice, but will it fit our needs? One reason to separate our View into SubViews is the possibility to swap out a subview to a different version. To check if the subframe data is sufficient to be stored, I would like to have a simple ViewModel per frame. Let's collect:

PersonView -> PersonViewModel -> PersonModel <- Database/Rest Interface

AddressView -> AddressViewModel -> ?
TelefonView -> TelefonViewModel -> ?
BankView -> BankViewModel -> ?

The PersonViewModel has no fields (or just two buttons).

Even if we do not store data in visual controls, we must define the controls. Lets name this Place B,C,D for the three subviews, and Place A perhaps for the PersionView that only has a clear and save button.

In the PersonViewModel we have to declare a Field/Property canSave (Place A1). Inside the PersonViewModel we must be able to check the three other ViewModel so every ViewModel from the subviews should be connected to the "parent" ViewModel.

In the AddressViewModel we have to declare the Field/Properties for the Address. (Place B1) And the same for the two other ViewModels (Place C1, Place D1).

In a best-case scenario, we would skip the Models for the subviews - if not we would also have Place B2, C2 and D2.

Is MVVM slower in development because you have to type everything three times?

Normally: Yes, it is!

Don't forget we're doing MVVM to get things faster developed! 

How can we archive this?

One major point for MVVM is Testability. Perhaps we are slower in the first place, but with a huge project and a lot of Models - if we have really good test coverage - we are much faster than other projects after a short period of time. Especially if you're doing TDD. 

If you haven't seen my first two videos on YouTube, please take a look. And also please subscribe to help me grow my channel!

My idea is auto bind by name, this saves you a lot of time. The other idea is now to store field values on the flow from component to the final database.

Let's take a look at the full development cycle and for simplicity name it in Delphi terms:

Create a TForm for the subframes, name it TPerson.View

Create a TForm for the address, name it TPersonAddress.View

Create a TForm for the telephone numbers, name it TPersonTelephone.View

Create a TForm for the bank accounts, name it TPersonBank.View

In the composition.root: Register the View's and implement the create for the Forms.

Create the 4 ViewModels.

As the ViewModels are "just" classes derived from TViewModel, we only need to implement a property for all components on the TForms. This is the place where we would like to store our values. To speed up things, we don't create local fields as 

FName : String;

we create this local field as

FName : TProperty<String>;

Auto-Binding will do the rest and connects the Name : TEdit from the Form to this "property"!

Create an empty TPerson.Model as TPersonModel = Class(TModel).

At this point - if you're doing TDD - I would create a TPerson.Model.Test.pas and also a TPerson.ViewModel.Test.pas. But this is a topic for a different blog post.

The base TViewModel class has a procedure call ViewModelSet that is called if the ViewModel is set to the View.

If this event is fired, because you set the property in the composition root like:

Result := TPersonView.Create;

LViewModel := TPersonViewModel.Create;

LViewModel.Model := TPersonModel.Create(TPersonDB.Create('local'));

Result.ViewModel := LViewModel; // here

By setting the ViewModel Property at the View, the Framework is doing the auto depending subframe load. So if everything is ready to go you have the ViewModelSet method that is called to setup the multi bind.

So we have:
1(3) : 1(3) : 1 chain!

If our MainView / MainViewModel wants to create our PersonView it just calls "Show('PersonView')" as part of the Navigation Service - Bang - everything will be created and everything is wired-up for the View and the subviews.

Now - how to transfer the Data from the View to the ViewModel to the Model to the DB and even from SubViewModels? Sounds complicated!

Let's compare RAD vs. MVVM!

In a "normal" RAD App you will have to define some kind of data structure to keep your data in memory. This could be a Class or a Record with all your field-values. In our case, there will also be some kind of TList<TBank> or TArray<TTelephone>. To get your data into your data structure you can use the live bindings, to a dataset or in the old days perhaps a DBEdit to do this the direct way. Really horrible, because while viewing the data, the database connection must be kept open or copied to a mem-table. Perhaps you have done it the old way by manual setting the values to the component and back by calling a method like:

FormToData, DataToForm where you have to set every field to the component.

EditName.Text := Data.Name;

EditStreet.Text := Data.Street;

// ---

Data.Name := EditName.Text

Data.Stree := EditStreet.Text;

Even if you use the live bindings this could get messy if you have many controls on a Form, and of course, it is horrible to maintain, because all changes are in the DFM/FMX file not so easy to merge in your repository.

Designing the Form is the same for RAD and MVVM.

Instead of creating a Class to hold your data, in MVVM, you create a ViewModel. (Perhaps some tutorial says that the data is stored in the Model, but this is not my approach). Creating the ViewModel should take the same time as creating your Memory-Class. The auto-bind will be faster the every other RAD approach.

1:0 for #D.MVVM.

Autoload of depending Frames would be 2:0 for #D.MVVM, but you could use real frames so this would not count. But setting up subframes in the composition root is easier to maintain, so let's call it even and don't count this. So we are still at 1:0.

Using the ViewModel for a Unit-Test to check if the Form works as expected: 2:0 for #D.MVVM. This should count 3:0 because Unit-Test with the RAD-Form is not really an option.

So for the design part, we are faster with #D.MVVM than with the pure RAD approach. 

And this is where the problems begin: We don't want to redefine all the fields in the model and thus double our effort and lose our time advantage. We also want to save ourselves the typing work and don't want to double our source code, which we have to maintain in parallel. But in some cases, we have to convert fields to be able to write them correctly into the database. The same is with the database itself. We don't want to create a separate definition for our database by copying the field from the ViewModel. 

Perhaps you use a tool to design your database or some kind of Modelmaker stuff to create your DB and Classes! From my point of view, this is not the way to go. I want to have everything in the source to keep track of it over the source repository.

How can we archive this?

Well, one idea is to use an on-the-fly created data context that is able to receive or set the property fields in the ViewModel and also convert the data-fields while sending them from the ViewModel to the Model and after that to our database interface. If we finally have a wizard in place for creating ViewModels and Models insight the IDE, it will be a piece of cake to do this. Without a wizard perhaps we have to rebuild the structure only one more time to create a universal data-structure that is able to deal with our ViewModel, our Model, and our CRUD-Database.

Without this approach let's collect our places where we have to copy all the fields. We need a copy in the Model (A2, B2, C2, D2) and in our database (at least in the SQL statement to create the DB and in every query to read or fill the values) (A3, B3, C3, D3) and as we remember Place X for the JSON.

This sounds like overkill and throws a bad light on MVVM. 4 Places to copy all fields... Horrible

That's why I do it in a different way with my #D.MVVM framework. Create your ViewModel, use the ViewModel definition, and some attributes to define an ORM-Like Model for the database and also for the Model, define some converter methods to handle or change field or field-types. That's it!

No need to define any fields in the Model: 2:0 for #D.MVVM.

Just define converter if needed: 3:0 for #D.MVVM.
Create the database with just one definition: 4:0 for #D.MVVM.

And don't forget this is VCL and FMX. The next step is a View to ViewModel wizard to convert your legacy VCL app to VCL-MVVM or FMX. 

This could all be done with the pure #D.MVVM Framework. Combining this with my FDK - The Firemonkey Development Kit, that contains many units and functions that are also usable with VCL, provides the intuitive MVVM developer with a tool that can not only create even the most complicated application with numerous interfaces in a very short time but also keep it maintainable for many years.

So stay tuned for the next #D.MVVM video on my YouTube channel!

#D.MVVM - At what point is a framework ready for release?

Besides some bug-fixes and the typical daily stuff, my focus is still on my Delphi-MVVM-Framework. 

#D.MVVM

Why, because I want to port all my old applications to this and I promised myself not to start a new project until the framework is ready so that I don't get the idea of either using an older version of it or not using MVVM at all.

There is also a dream, that I'm able to port our 35 years old main VCL - Application from D2007 direct to 10.x - #D.MVVM - at best direct to FMX.

So, I have to make sure that I can call the framework finished, which means I also have to set the needed functions to the status "finished".

Of course, we all know: With such a project, as with any other application, there is never the status "finished". So how do I find out when I'm just "tweaking" it? Ok. first of all, I need a feature list. I would really like to have a unit test for each feature. Second I need sufficient documentation. (Oh, boy this is a breaking rule. I have no documentation at all - so far).

But what's about clean code - or at least cleaning up the code (a little bit)? Yes, this is 100% necessary. 

How do you find out, if an entry in the feature list is a really necessary feature and not some nice functionality I just want to add? I really like to read your comments about this. What are the core functions, you think must be included for a 1.0 Version of my framework?

Before we try to collect some features we should define some rules.

To implement an application with the MVVM-Pattern we want:

1. No, or nearly non-code in our View! Except for UI stuff!
2. Comparing a RAD-App with an MVVM-App, MVVM-App should be as easy to create.
3. The bindings are the biggest problem, they must be as easy as possible to set up.
4. No use of Anti-Pattern. (Some people are calling a ViewFactory already an Anti-Pattern)
5. At best: Just create both kinds of  "Forms" and you could compile it to VCL or FMX with the same codebase.
6. Every part of your application should be unit-testable, so no dependencies between your modules. 
7. Don't break the layer concept.
8. The framework should work without the need of the FDK. (I had to duplicate some stuff)
9. D.MVVM integrates as a plugin to the FDK to benefit from all the higher-level functions.
10. Don't use the visual live bindings.
11. Everything should work, without clicking non-visual components or special components on a form.
12. Naming conventions over configuration.
13. Don't care if a C# developer thinks you are doing it wrong! 😄

Anything else?

If you know me, I really dislike clicking things "together". I like to do everything in code - except the form of course - so that every change is noticeable in the source-code repository. Perhaps I create some components at the end, to fit the needs of the RAD-Development, but that is not yet decided.

While creating a feature list, I can also create some tickets to keep track of the development process.

My D.MVVM Framework should contain:

1. A Wizard to create a new D.MVVM Application.
2. A Wizard to create a new View (Form or Frame).
3. A Wizard to create a new ViewModel.
4. A Wizard to create a new Model.
5. A Framework (VCL & FMX) independent overload of the TForm implementation to give every Form some base functionality. (done)
6. A central point to do all the joins or Viewchains. (done)
7. All included VCL and FMX components should be bindable. 
8. 3rd party components should be easy to include in the binding schema.
9. Bindings from the View to the ViewModel should be possible without writing any line of code. (done)
10. Bindings from the ViewModel to the Model should also be possible without any code. A value converter should be included in the data transfer event. (done)
11. A data context should keep the data in memory to be displayed. (done, but not fully tested)
12. Some basic services should be included. (Action-Service, Navigation-Service, Menu-Service, Dialog-Service, Time-Service) 90% done.
13. A List-Adapter for handling huge Data in Grids, Listboxes, and Memos. (done)
14. Public Unit-Test with 100% code coverage. (Some are done, but I have not measured the coverage)
15. VCL and FMX demo Apps. (some are done)
16. Documentation.

I think this is not the final version 1.0 list yet, but almost.

Again: After watching Uncle Bob's clean code lessons - I have to do some heavy refactoring...

If you have any suggestions for me, I would be very happy about your comment on this topic!

If you're reading my blog for the first time, please take a look at my Youtube Channel for my already released D.MVVM-Videos.

Please don't forget to subscribe and hit the like button to help me grow my Channel.