Monday, May 3, 2021

TCoffeeAndCode - Threading and Concurrency

Hello!

Are you interested in learning more about threading and concurrency?

On Wed, May 5, 2021, 7:00 PM - 9:00 PM CEST 

or for better understanding 17:00-19:00 UTC,

you can listen to how we approach these things and what our opinions are on these topics.

Please Register and save your seat - here!

Saturday, February 27, 2021

Delphi 10.4.2 - What is your current Hardware for Development?

Well, this is of course not related to this newest version of Delphi, it's the right question for every Developer, but with every new version of Delphi, this topic is back on the table. If you're looking for my opinion on 10.4.2 - please just scroll down!

Declaimer: This is not related to Embarcaderos specifications of minimum requirements, this is "just" my point of view.


Let's start with a little example:

If you were a carpenter, would you buy a jigsaw from the hardware store for 49.95 or would you have a high-end circular saw table that can do everything you need?

If you have to cut down an entire forest, you wouldn't use a handsaw, but a chainsaw, would you?

Is it possible that your 3D printer cost more than the laptop on which you "work"? 

I just read a question about the new IDE: "If I press a key and the LSP has to update, I can hear my Laptop-Fan is ramping up... 

Laptop? Really? Are you kidding me... 

There are some Laptops out there, you can probably use...But for this price, you better buy a third Monitor.

Ok, If my system is mining in the background and I'm watching a Video while loaded up 3 VM with different IDE's sometimes I hear the fans of my watercooler spin a little bit faster. Just joking, minding on the GPU is stupid.

Your Workstation is your main tool for work, yes I call it Workstation, Powerhouse, whatever... If you call it "My PC" and you are not able to run the latest AAA games on it. Buy a new one!

After 35 years of Development here is my list of NO-GOS (in unsorted order):

  1. Use a non-SSD Harddrive or an SSD attached via USB.
  2. Not installed the latest OS
  3. less RAM than 16MB
  4. Screen resolution less than 1920x1080
  5. Only one Monitor.
  6. Not using a VM
  7. Using a non-standard keyboard. (perhaps you can work on a Laptop keyboard I could not)
  8. Not using a Source-Code-Management System (svn, git, mercurial)
  9. Firemonkey App development without my FDK... ;-)
So what are my recommendations you ask?
  1. I would never want to develop without a VM again. So: Use a VM. I like VMware, there are some issues with it but overall it runs perfectly. Using a VM set the bar for a good system a little bit higher! But having auto-snapshots every hour and be able to revert the installation back to a working copy if you have installed something stupid or your installation just broke down is worth it.
  2. Your CPU should have a core-count that is double as high as you would like to have in your windows system. I prefer 4 cores so with hyperthreading you have 8 logical cores in my windows. That's why my CPU has 8 cores with 16 logical cores. 4 for the VM and 4 for the Host. I'm an Intel-Fan-Boy, never tried AMD. My colleague has bought a board with the latest AMD CPU and he has so many problems...
  3. Use 3 Screens. 
    1. Center screen for the IDE. 
    2. Left screen for the undocked the Objectinstpector and structure view. 
    3. Right Screen for the program group and the component palette. 
    4. My Line for breaking up the source line is not at 80 anymore, it's on 160!
    5. While debugging, place your program on the right screen (program group not shown). 
    6. Most of the time when doing boring stuff I put my OI docked at the left side of the IDE on the center screen because I switch my left screen to my Host for watching YouTube videos or other stuff...

  4. If you want the "same speed" inside a VM that you would expect from your main system, Harddisk and RAM performance is the key.
    1. I've installed 64GB of RAM so I can start 3 VM's with 16GB and still have 16GB left for the host system. Or any other combination. 
    2. Perhaps you think: RAM-Speed is not so important. I did not test this and perhaps high-speed RAM has a not-so-big impact on your system. I prefer RAM modules that are recommended by the Mainboard Manufacture and often higher speeds come with a huge impact on the price.  But this looks not so bad compared to the rest of the world.


    3. On the Drive-Side there is no room for low-cost at all! Maximum performance is the key to a smooth-running VM because the VM is often very disk intence.
      1. You get the best performance with an M.2 Drive. If you combine 2 or 4 of these high-speed M.2 to a RAID 0 stripe-set you are pretty close to the maximum speed your bus could handle.
      2. Never ever use your VM on a Harddrive or a single SSD connected to a SATA connector. I had SSD failures in a RAID 10 and this saved my installation.
      3. For Backup I have a RAID 10 of 4 SSD's and a RAID 5 with 3 M.2! Here is the trick: I use the hourly auto-backup of VMWare. That's fine. From time to time I shutdown my VM and copy it (1,6 TB) from a RAID 0 (4 x M.2) to to the RAID 5. This is the best performance I could get. After a few minutes, I can restart my VM and then copy the files for this RAID over USB C 3.1 Gen II to an M2. in a fan-cooled external housing. With this, I get the maximum performance USB could handle. I really would like to have a Thunderbolt connection (should double the speed), but the card I bought was not working and I have no spare slot on my Mainboard. 

  5. I know some people prefer Air-Cooled systems. But I'm using a custom loop for so many years now and the only problem I had once was because my soft tubes are getting old and one fitting was leaking. My System runs quiet, always below 50° C (often below 45° C) and with adding a little bit of water from time to time I had no maintenance to do. (Yes, I just use water, not any colorful additive or even distilled water) That's why I have no little particles clocking up my block. With my last CPU/Mainboard update I installed a new reservoir (the old Plexiglass had got some cracks) and switched to a new pump (just in case). This was the only maintenance in 10 years. 
  6. Install a really good graphics card. Perhaps not so important for VCL developers, but as FMX uses the GPU to render, you'll get the speed you've never seen before with VCL.
  7. For the App development, I'm using a Mac-Mini. Ok, I switched out the hard drive and installed an SSD. (I really should test a macOS-VM sometime). 
  8. And similar to point 9 of my no-gos - use my FDK for your FMX, VCL, and App-Development on the mobile platforms.

Anything else?

hmm... I'm still on D2007 with my main application. (Not finding the time to fix all the Unicode warnings). But the old compiler is also able to compile more than 2. Mio. LOC in under 16 seconds for a full build.
 
Perhaps you call my recommendations overkill, I call it: Best bang for the bug... I spend so many hours at my workstation and I have no time to wait for something to finish. That's why I want the best performance you can buy. (For a reasonable amount of money).

I love the statement about electric cars from the best rally driver of the world Walter Röhl: 
I'm too old and I don't have enough time left to wait at the loading dock. (Translated)

In my case: I'm too old to wait for the compiler. If I currently working in a unit to implement something and I have to take a look into a different unit (normally I have too many open units, often more than 20), it is faster to hit compile and let the compiler find the position I am working on, then to search for the unit or ever do a text-search inside the same unit... ;-)

So - don't complain that something is too slow, beef up your system!

If EMBT tries to speed up the IDE and the compiler at the same time, we will get a faster and faster development environment or at least a constant speed as the possibility grows.

Back to 10.4.2


In my tests 10.4.2 is faster than 10.4.1 and 10.3.3. There is of course an impact now many components you have installed and how long your search path is. Also, there are differences when compiling inside the IDE or using the external compile or MSBuild.

Besides the numbers, the IDE feels snappier and there is a lot less flicker (or nearly/no flicker anymore). There is a little loading screen, popping up when the IDE is loading units. You can disable this, but for the first time you get a notification of what the IDE is doing - I love it.

There is also this new feature of the LSP. You know the old red wriggling lines below errors that were wrong in most cases! With the new settings, you could switch to different lines or dotted lines that also could have different colors. Blue, Yellow, and Red for "not used", "hint" and "error"... My first reaction to this was...

Why on earth did they do this?

But while using this feature for some days - I love it - you can faster correct your mistakes or change a var name because you misspelled it.  This allows you to develop faster, believe me.

Overall it feels better - yes there are still some little bugs in the IDE - we all know, but I can live with the new version!

And as always:
We hope for the next version (10.5) and that the little bug gets fixed!

So long, have fun with the new version.


Friday, February 5, 2021

My road to a useable MVVM Pattern implementation for Delphi!

If you read my blog regularly, the MVVM topic will be nothing new to you.

But please take your time and fill out my new survey, thanks for that, of course, even if you visit my blog for the first time. I would love it if you can also be part of this survey

If you are a first-time-reader... Welcome to my blog. Please read the other posts about my #D.MVVM implementation. As you can see, this subject is not new to me and my knowledge has, of course, improved over the years. (I hope, I'm right with this statement)



Here are the links to my other MVVM-related blog-posts in reverse order:

There was a blogpost from Oct. 2019 with the question: How long does it take to develop a "complete MVVM framework for Delphi. (Link above) the answer is: Not long if you take shortcuts or too long if you are a single developer with no help. But I won't cry, because my goal was not to do it the easy way.

If you do not make it down to the end:
Here is the link to my new survey!
But perhaps you want to know why I've created a new one? ;-) 


Let's collect - one more time - my goals:

  1. The framework must be as fast as possible. (My goal is - it is supposed to be the fastest MVVM framework available for Delphi.)
  2. The development time should not be significantly longer than using the RAD-Approach. At best, the development time should be shorter, comparing to a non MVVM, but not "everything in the Form unit" - development.
  3. Everything that could be done in a background thread should be done in a background thread.
  4. Support for FMX and VCL. Including the "special needs" for the FMX UI.
  5. It must definitely run on ALL platforms.
  6. 100% Unit-Test-able of your Application. (Switch for disabling the background thread to do a better Unit-Testing).
  7. Mockup's for View-Components to test the Binding of your App.
  8. A nice Wizard should be installed into the IDE to create a new VCL or FMX-MVVM Application. This Wizard should be changeable over an Ini-File to make a User-Configuration without recompiling the Wizard-DLL.
  9. One central binding point (Composition.Root) 
  10. If you are an FDK-User (My Firemonkey-Development-Kit) the framework should work streamless as a plugin, with all the benefits of my ORM-Lite, threading, or Client-Server units.
  11. No need for special components. (Components must work without an extension)
  12. Open to User-Implementation for non-standard components.
  13. The View (Form) should not have a single line of code. Or should not need any code-behind...
  14. Visual Livebindings must not be used.
  15. A developer who has NEVER dealt with MVVM has to get along with my framework IMMEDIATELY.
  16. A good developer who has already used MVVM should find his way around immediately. (Except for some things that I have implemented differently than MS).
  17. We are Delphi-Developer - If we are doing MVVM differently than our C# friends - that's fine!
  18. Views must be able to be composed of any number of sub-Views (frames) without writing a single line of code for them. (Sub-Views must be interchangeable for special requirements)
  19. It must also be possible for a sub-View to be a ViewPort3D (FMX-Only). Full support of 3D objects from the ViewModel will be another thing for further investigation. This could be very interesting for Game-Development, but that is completely another topic.
  20. User-Level and flagged Menu-System with auto Action-Binding.
  21. Multiple View for a ViewModel, also with sub-View/ViewModel Binding.
  22. Conventions over definitions.
    1. This is especially true for automatic binding by naming convention. Example: Name: TEdit should auto bind to the fName: TProperty<String> or to a normal property Name: String. The Binding is connected to the Text-Property of the TEdit Component and the OnChange and OnChangeTracking (FMX Only) event should update the ViewModel.
    2. User-defined naming convention... Bind Name:TEdit to fEdNameField : TProperty<String>
    3. Global naming rules and also naming rules by TypeInfo.
    4. Overwrite any predefined rule with an attribute.
  23. Multi-Binding to different properties.
    1. Example: Save: TButton.OnClick -> Procedure DoSave
    2. Save :TButton.Enabled -> fCanSave : TProperty<boolean> or Property CanSave : boolean
    3. Fone : TCombobox.Items -> fFoneItems : TListProperty<String>.
    4. Also user-defined naming: "Can"+Save;"Do"+Save could be changed to: e.G.: -> "CheckFor"+Save;"Execute"+Save.
  24. Optimal performance on ".Strings";".Items";".Lines" - never ever send the complete list on a single change. A virtual created adapter must take care of the updates. (e.G. for a single changed line in a Memo of 10.000 lines it is a nogo to update the Text-Property of the Memo.)
  25. Data-Verification on Property-Level to check the fields inside the ViewModel.
  26. Listview and Listbox special treatment for FMX.
  27. A data model for VCL Grids - The FMX Grid already has a data model.
  28. A not so easy implementation of an MVVM-Treeview. (Perhaps not in the Beta)
  29. Complete logging of all internal functions should be possible. (Only for the Plugin Version)
  30. Error messages that are normally written in English must be able to be translated by an external language file.

The MVVM-Pattern has the Model included in the Pattern-Naming, but actually, the Model is not really a part of the Pattern. But in my case:

  1. Only if the Framework supports the transfer of all the collected data from the ViewModel to the Model, my goal of: "As fast as RAD Development" is possible. 
  2. The Framework must have a Datacontext that can make the transfer from the ViewModel to the Model, and back.
  3. The Datacontext should be able to convert, combine and separate fields as they are transferred back and forth.
  4. The Model should directly work as an ORM or REST-full Web interface. 
  5. As an FDK-Plugin it should also work with my Client-Server-REST-Units.

So far so good...

There is light at the end of the development tunnel! I'm really happy with the actual implementation. I think I have done all the necessary improvements for the first version. and - as always - I would have programmed the framework even if I wasn't planning to offer it commercially. Of course, programming is a bit different when the source code is handed over to other developers. Certainly, I could have made a lot of things more static or simpler.

And now?

There will be a Beta-Version and an Early-Bird-Order! But maybe this time I should do an Alpha -Version too - if I can find developers interested in helping me finish the release version.

But how and why?

After years of development - only in my spare time, of course - perhaps I've done something wrong, perhaps I should rename a method or change params of a function. Whatever... I don't want to make breaking changes from Beta to Release.

I know - some of you - are waiting for the release of my Framework, but besides the old survey, I don't know how many Developers would still like to be part of the Alpha or Beta-Team. (or even would like to buy my #D.MVVM-Framework). (Old survey filled by 177 Users).
I've talked to some of these developers and they can't wait to get started with my framework. (I'm super happy to hear that, of course).

A comment in advance from René Höger:
"Is a lot of information! Those who have dealt with this, I think will be hot on it! I don't know what your feedback is so far, but I think this could be a milestone. When I think of all my old applications! But maybe I am biased because I have already received an introduction into your framework!" 

💖 Thanx René!

Perhaps I need to write some kind of NDA contract (never done before). One thing I want to avoid, after all these hours, is to find my Source-Code as a zip on a hacker site.

Before I could start the Alpha-Phase, I have to:
  1. Open up a ticket system or some kind of bulletin board?
  2. Perhaps a discord server?
  3. Perhaps a slack group?
  4. Think about how to collect User-contributions...
  5. Think about contribution benefits?
  6. Late or discount payment for Alpha & Beta users? 
How do I distinguish between a developer who "just" wants to see cool source code and a developer who wants to actively contribute to the framework? Maybe I should just assume the best and trust that Delphi developers are all super nice people who are interested in using this technology. I think I'll stick with that idea or a very little "fee" just to check if the developer really wants it? Perhaps some kind of reward system?

If you've made it down here - Thanx for reading and please be so kind and fill out my survey!


Saturday, January 23, 2021

The IFDEF Problem!


If you are using {$IFDEF} in your source - normally it's fine...

e.G.

{$IFDEF DEBUG}
ShowMessage('Foo');
{$ENDIF}


Why, because this DEBUG is handled by the IDE.

But what if you want to use your own?

{$IFDEF IMPORTANT}
Result := 42;
{$ENDIF}

If your application depends on this, you always have to do an else part to make sure the Name "IMPORTANT" is set.

{$IFDEF IMPORTANT}
Result := 42;
{$ELSE}
! Too stupid set this!
{$ENDIF}

I don't want to have this in my source in every place I just want to exclude or include something.

Like:

{$IFDEF UseLogger}
Log('Ready to run');
{$ENDIF}

In this case, I want to be able to include logging, but on the other hand, I want to be sure that I disabled logging for a good reason, not just forgot to set the {$DEFINE UseLogger}.

That's why I like to use const's:

Unit Defines;

Interface
  Const
    UseLogger = true;
Implementation

end.

{$IF Uselogger} is also not working if you just forgot to "uses" your Defines.pas Unit.

That's why I'm using the not implementation.

Unit Defines;

Interface
  Const
    NotUseLogging = false; // means use logger
Implementation

end.

Because using:

{$IF not(NotUseLogging)} has the benefit of checking if there is a symbol named NotUseLogging!

So if you forgot to use the Defines Unit you are getting an error at compile time. I'm using the approach for my FDK and also for my #D.MVVM Framework.

This works fine... But there is one drawback: You are unable to change these settings per project. In the past, I've changed the source of my defines unit, but changing this back and forth is a pain. Of course, this unit is named Delphiprofi.FDK.IFDEF.pas!

The other drawback is, you are unable to use the {$UNDEF Foo} in a Unit. So what is the best solution?

Let's collect the necessary things we want to accomplish:

We want to be sure to set important things globally for the whole framework. That's why I use the {$IF Not(Name)} statement. In the past, we all used this stupid {$I defines.inc} approach, but I hate that so much and it also has the problem that your {$IFDEF Foo} is unable to check if you just forgot to include the file. How often have you searched for a problem, because you misspelled a conditional define? I always type  {$IFDEF WINDOWS} instead of   {$IFDEF MSWINDOWS}.
Of course, we can still use normal {$DEFINE Foo} and {$IFDEF Foo} for local unit settings during development. While using the "if not trick" in every unit we are sure that all {$IF not} statements have the right setting... 
And how to use these settings on a project level? Of Course with a normal {$IFDEF} in the Delphiprofi.FDK.IFDEF.pas Unit. In my case, I take this idea one step further by using a class like:

ExcludeFDK = class sealed 
  public
    const
     Logging     = false;
     //... more in the original source
end;

With this, I just can type ExcludeFDK "." and the IDE gives me all possible settings (with the right spelling).

The If looks then like this:

{$IF Not(ExcludeFDK.Logging)}
FDKConfig.GetLogger( Self ).Warning( 'This (%s) should not be happend',[aMSG]);
{$ENDIF}

btw. The FDK is also able to disable logging, based on the Type of the Class the logger is fired from. If I want to disable all the logging from my DI - Container, I just set the exclude of this class in code. Very handy!

To be able to include or exclude - in this example - the logging from a project I can use normal defines on the project level. In this case, it is safe, because I know that all units are set correctly. This it how it looks:

ExcludeFDK = class sealed 
  public
    const
     {$IFNDEF FDKNoLogging}
     Logging     = false;
     {$ELSE}
     Logging     = true;
     {$ENDIF}
     //... more in the original source
end;

If I have no conditional defines in my project settings, the behavior is the default. But by using FDKNoLogging all the logging functionality is not linked to the project.

But this is only necessary if I want to exclude the logging completely from a project to reduce the size. If I keep the logging linked, but do not set a logging target, a NIL logger (empty procedure call) is called with the least possible CPU cycles. So you are able to activate logging - for example - over a command line param.

If you want more information about the logging features like Log to Console, Debug Window, Testinsight, SmartInspect, or to a remote server or any of the other targets, please leave a comment.

That's it for the moment... More is coming...

Friday, January 15, 2021

JWT - The JSON Web Token!

Sometimes I use my free time to discover something new (or new for me).

This time it should be the JSON Web Token.

After the first consideration, I didn't understand the use of it at all... 

I (the server) create a token (Header. Payload), make Base64 out of it, and add a signature by hash with a key. 

OK, so far so good. If I send this token to the client, the client can take a look at it with a Base64 decode. 

(Why should I allow the client to do that?).

The client should send me this token with the next request and I (the server) can apply the hash again and check (the client doesn't know my secret key, which I used for the hash), if the hash matches, that is the way I can check, that it is still "my/valid" token. With the payload, I can then identify the user.

Fine - where is the use case?

The server generates the token after the user-login and doesn't need to store this session token. Cool.

How else was it done in the past? 

Generated a GUID, add a few random bytes, then maybe a cipher, and put this into a database. Next time you could just access the database with the token and after the access, you had everything you needed from the user. And the Client does not know anything about the token or what is stored inside - I prefer this, but:

With the JWT, the session database is not needed and the payload of the token can be used to load the user information. Perhaps for your web application, this simple "User is Valid" information is enough! 

The structure of the token is like this:

Base64(Header) "." Base64(PayLoad) "." Base64(Hash("Header.Payload",Key))

Header and Payload are JSON strings. Of course, someone has made countless thoughts about what has to be contained in the JSON. 

So the header could look like this : {"type": "JWT", "alg": "HS256"}

Token type and hash algorithm.

There is also a rule for the payload (RFC 7519). This may all have a justification and looks like this:

{"sub":"Frank Lauter","iat":1610651797,"exp":1610738197,"iss":"FDK-Token"}

Subject, Issuedat (Timestamp), exp (Expiration Time), iss (Issuer) e.g. the library used.

The algorithm works of course also, if you write completely different JSON in there.

One trick you have to keep in mind is that not all characters of the Base64 encoding can be used as URL-Querry-Param. Normally you would think that just a URLEncode is called because it makes the usual %HexValue stories, but this is not so and also the "=" at the end must be deleted. 

Facebook also uses this coding... As Marco wrote 2012!

If you google for "JWT and Delphi" the first link will be a github repo. 

These 24 units - 173KB source - work very well! ;-)

Perhaps you will like the source and if you have to address all the RFC-Rules you might want to use this... 

Or you just write this few lines in your Webbrocker application:

Yes, you have to insert the DateTime JSON Value, and I set it fix to SHA256 bit. 

You need


If you get back the token with the next call you just use:

for this you need the URLDecode:

You can of course improve this little demo and support different Hashes, or use your own content and give a shit about the RFC's - My Server, MyApp, My Rules!

Have fun...


Sunday, January 10, 2021

Outside the MVVM Pattern?

In the past, I always thought the Model is a class closely related to the MVVM pattern... However, this is not really the case. The model should be able to send notifications or must be able to do some data mapping, but in a real-world application, the Model should be YOUR Class - perhaps implementing some interfaces if necessary.

But this is not the topic for today!

Today I want to talk about colors and other effects. Spoiler: The ViewModel has no knowledge about colors. Neither does the Model. But why do I want to set colors?

To indicate a status, of course. For example, an input is incorrect or that it is a required field. Maybe in such a case, I want to switch the background to red or show a small asterisk next to the input.

So actually I don't want to set a TEdit.Background to red, but just visualize the status.

But what about Visible and Enabled? These states could be part of the view control. Do I make an exception for this?

Sure, I think this is a good idea. Already at the beginning of my MVVM series, I said that I want to adapt the pattern to Delphi and of course to Delphi users and as I know, we all love to disable elements or set the visibility of a whole panel to false, because we need the controls on the panel just in some special reasons. We could do better, but I think I'll make this exception.

If my ViewModel belongs to a RichView, it looks different, because here all font attributes are possible, but this special case I don't mean here.

So how should I send the status to the View? I don't want to create a new label for everything. I'm thinking more about the implementation of Bootstrap or similar systems. There is no color red, it's the status warning, same as not yellow but status is hint. This status can refer to the whole View or only to one or more controls. For this, I still need a nice pattern and possibly a singleton for all views or view groups, so that you can store your own settings.

For verification I dislike the RegEx approach, I would normally call a method. Perhaps the Method is also not in the ViewModel - depends on the problem. In this case, I would trigger the ViewModel and the ViewModel could ask the Model. As everything is just a call for "get status", you can do everything you want with this. This should also be possible from a background thread (perhaps it's a REST or DB request).

I have already implemented a DeepSetter to enable the ViewModel to set everything on the View with a change event. But this was just for the proof of concept. Because if you use this kind of call, you are perhaps unable to change the View. (Bad Idea)

Let's do a list with a bind and not bind. For a TEdit -  of  course, you can bind to everything in code-behind or with an event-token-call, but this list is for the default bindings.

Align - not bindable
Cursor - not bindable
Enabled - bindable
Height - not bindable // But what if I have to resize the control on content?
Hint - not bindable // But perhaps connected to a translation table?
MaxLength - not bindable // Tuff desicion
Password - not bindable // Should be handled by the View (Button show PW)
ReadOnly - not bindable // Fix by design
StyleLookup - not bindable // Fix by design

Perhaps the StyleLookup is the entry point for the status info.

TabOrder - not bindable // Fix by design
Text - bindable // of course
Textprompt - good question // I will decide later
Textsettings - bindable to global Textsettings?

Visible - bindable
Width - not bindable

Now to the Onxxxx Events: (Everything could handle by the View)

OnCanFocus - good question // I will decide later 
OnChange - bindable
OnChangeTracking - bindable

OnClick - not bindable // for a TEdit - a TButton would have this event.
OnDblClick - not binable
... // Skip the list of not bindable events here
OnValidate  - good question // I will decide later 
OnValidating - good question // I will decide later 

Please Remember: For everything not bindable - this is just for the default setting. You are the Developer and if you like to screw up the pattern you could bind to everything.

So what do we have outside the MVVM Pattern?

- Enabled
- Visible

The Status is well inside the MVVM Pattern like:

- TStatus = (Valid, NotValid, Warn, Hint, Information, Mandatory) // Perhaps more?

The View could ask the StyleView-GuideLine-Singelton for display colors based on the TStatus.

That's it, are we done?

It will be a time-consuming process, but I will provide some examples with the #D.MVVM-Framework to demonstrate all these things.

I can't wait to convert my first 780.000 LOC legacy App to #D.MVVM.

Have a nice Week...

PS.: If you've made it so far - Spoiler: Next step is to open the alpha user list. If you are a Delphi 10.1.x - 10.4.x User (VCL & FMX) I will provide a link where you can submit your request.


Friday, January 1, 2021

Happy New Year!

 Hello, my friends!

I hope you are all healthy and have a safe job in these crazy times. The year is over, but I think we will still have a long time of "fun" with it.

For the new year, I have clear priorities. (Besides my main job of course)

1. I want to get my #D.MVVM framework up and running as soon as possible and then release it as the first alpha version.

2. The new version 2.0 of the FDK lay, unfortunately - not at least because of these crazy times - a little bit untouched on disk. Of course, I made some bug fixes and added new features. Not everything I wanted to implement is ready yet, but the #D.MVVM framework has priority because it is the base for all new and old projects. Because such a framework as my FDK, actually has no clear version jumps, but rather a continuous improvement, there is no reason to wait. So if you are interested in my FDK, you could order the current version. Updates will still come.

3. Youtube. I hardly found the time to create a new YouTube video in the fourth quarter of 2020. In addition, unfortunately, the Vol.03 of the D.MVVM series (45. Min.) is destroyed. (Sure I have a backup, unfortunately, the file directory structure of Camtasia is completely stupid and does not collect the necessary clips in one place (or at least there is no function for this), but can use links across all disks and network drives. Unfortunately, there is no warning when saving and so you can easily overlook that large parts of the clips that you dragged into the project are outside the file structure or in temp directories. Actually, you should create a zip and reload it every time you save - this is also the workaround that Camtasia gives).

Actually, I would like to do a live project with the Apocalypse Coding Group again, but we can't agree on a project at the moment.

This brings me to the topic: Comments!

Your comments on Youtube videos or also on my blog post would be very helpful to adapt the topics to your wishes. Of course, I am happy about every subscriber and every reader of my blog, but your comment and a suggestion would be helpful!

So get to the keys... I wish you all a successful new year. May the virus not be with you.



Wednesday, December 23, 2020

XMas time is Unit-Test time...

Give yourself a gift that will last longer than New Year's Eve. In many cases, you will be happy about it for many years.

There are still some grinches who don't believe in the satisfaction of unit tests and think: It's just a myth told at developer conferences.

To be honest, I program too few unit tests, but enough to find bugs I would never have discovered otherwise.

Just yesterday, a small change in my FDK would have caused numerous of my applications to stop working. However, since I use Stefan's Testinsight I got immediate feedback from my unit tests when I hit save... 

Here for example:

before:

Function FindSomeThing( const aName : String ) : boolean;
var
  lShelp : String;
begin
  Result := FindField('f'+aName.ToLower,
                          aName.ToLower,Nr,lShelp);
end;

after:

Function FindSomeThing( const aName : String ) : boolean;
var
  lShelp : String;
begin
  lShelp := aName.ToLower;

  Result := FindField('f'+lShelp,
                          lShelp,Nr,lShelp);

end;

My goal was just to do the .ToLower once. I already had this helper string for a not used param, so where is the problem?

Five tests failed after I hit save.

Let's have a look at the FindField method.

Function FindField(Const aName1, aName2; out ListNr : Integer; out Name : String) : boolean;

aName1 contains only the "f" and aName2 is empty, because of the out param. Did you know this "problem"?

So use the quiet time to write one or some unit test, maybe for a routine that has always caused problems or just to get a better source code coverage.

And if you haven't found the time to deal with unit tests yet, then use this quiet time and try it with simple examples.

Merry Christmas and a happy new year.

Frank

Tuesday, December 8, 2020

Create your Website content with Delphi!

Every time I have to create some kind of Webpage, I have to decide how to build the dynamic content. 

This is a follow up of my other blogpost about: "How to write you own Serverside-Extention!"

In the old days, it was easy! You had to provide a Webpage with a maximum horizontal resolution of 800px because this was the most used "device" (PC) resolution used by the visitors.

To keep the navigation of each page in only one place, your main design was a framed page setup. I still love this concept and I really dislike this horrible "scroll down to the earth core" pages. Of course on a smartphone, the frame concept does not work anymore.

To build a nice maintainable page you can use ASP.NET. With ASP.NET it is easy to create a master page and content pages. The server takes care of combining these two (or more) files at runtime, in nearly any other case you have to copy the navigation to each page.

Btw. if you like PHP, don't talk to me... ;-)

Creating webpages with ASP.NET and Delphi is nearly the best you can do. You have the JIT compile that creates your background binary code for direct execution, you have "native" code that is executed by the IIS and your pages are super fast. No need to stop the IIS to change the DLL or anything else.

Too bad EMBT/CodeGear dropped the support of ASP.Net after D2007. Yes, the HTML designer was bad and after D2007 there was prism. I've done some pages with prism, but this is not the same.

There is another thing called VCL for the Web. This works with Delphi forms and really long state data for the back button. This works also fine but is a completely different thing.

The other possibility is a Webbrocker application. This could be used on Linux and Windows. With a loader that is able to unload and update your DLL (Windows) you can easily update to new versions. 

If you want more information about this topic, please follow the link above.

The IIS will create a new thread for the request and executes your DLL. This is fast as hell because this is a really native CPU execution of your code. The final content as a string or a result stream is handled from the IIS and transferred to the user.

With this kind of DLL every single char/byte is streamed from your Webbrocker-Application and you have 100% control of the final result. You can load a page from the disk and process it - change tokens to different content or create a complete table from a database. This is fast, that is not the problem, but often you have to provide extra HTML content to format your final result and this is a little drawback.

Your Webbrocker application could also be used as a RESTful Webservice and the result could be XML, JSON, or SOAP, but this is not the topic for today.

For example: If you want to present a list of persons nicely formatted, you could change a token <#PersonList> in your HTML-Page to a list of persons from the database.

Let's ignore for the moment that most HTML-Editors dislike this kind of token.

You have to format the content with additional HTML-Tags like <br /> or <ul> and <li> or even a <table> with CSS. If you have a page that is static for a long time - no problem. But with every change of the presentation, you have to change your source code, recompile and upload a new DLL.

You could use an async call in javascript to your DLL for a REST-Call and process the JSON result also with javascript to the final HTML, but this is horrible from the viewpoint of a Delphi developer.

So what to do?

For many years I've used the Webbrocker approach and it works for me. This time I have the first project where the Webdesigner is unable to use Delphi and the Delphi developer has no clue about Webdesign or Javascript.

Well, Delphi Webscript (DWScript) is perhaps one possibility, but this is much too Delphi driven.

After some research, I found a project from Marco Cantù implementing Razor. Razor is also for C# and ASP.NET, but Marco's implementation is using the Webbrocker to process the HTML-Pages that contains the Razor @ Syntax.

This is really nine, the syntax is given and the HTML-Editors do like the tags with @Whatever more than the Webbrocker tags. The implementation for loops is "Javascript-Like" and it is very easy to include HTML-Tags into a loop for displaying data from a database or a simple loop. So the Webdesigner is able to change to look, without a recompile of the DLL. One thing I dislike with this approach is: 

  • You have to compare every byte in the source HTML to find the @-Tag
  • You have to syntax check the HTML-Source
  • You have to replace the @include Tags after all the processing.

Perhaps your server is fast enough to do this byte by byte comparison for every request in real-time, but this is against every rule that is deeply implemented into my development soul. 

Perhaps I should implement a converter to process the HTML-Pages and create a jump-table to do the trick? With a jump-table, you do not need to search for the @ token... There is still the include page tag problem, but if the included page has also a jump-table is more like a "just-stream-copy" thing than processing a string replacement...

After one day...

I call it a JIT-Compiler.

If the Webbrocker wants to load an HTML-Page I have to check if there is a ".htbin" file with the same timestamp. So I use findfirst not fileexists to check this (with findfirst I get the SR and the Timestamp). If there is my bin file, I'm done and I can use the prechecked bin-format -file with the Jump-Table, if not I will use the normal processing for the first time and create the bin file on the fly for the next request. This is really fast and I like this idea.

Should I include a Delphi-Script tag to process Delphi source? Perhaps in the future.

Do you want to see a Webinar for this? 

Do you want to use this for your own projects? Should I do a git fork of Marco's repository and put this on github?

Please leave a comment...

And as always: Please visit my youtube and hit subscribe to help me grow my channel!




Monday, October 26, 2020

Setup your Bitbucket replacement for your Mercurial repository

A long time ago the German DelphiPraxis community convinced me to abandon my previous strategy of saving my source code via zipping only and use a source code management system.

So I googled a little bit and found SVN.

After some days of getting used to it - I proudly announced: "I have switched to SVN"! And with that, the shit storm has begun.

Why aren't you using a distributed revision-control-system? Don't use this old SVN system! Uses Mercurial or Git!

So I took the first "Mercurial" and with a little help and a talk from MVP Uwe Raabe on one of the Delphi-Tage-Event,  I had to realize that this system "just" works very well and offers all the advantages I missed in SVN. ( Sorry Craig )

Pushing everything for backup to a network-driver or a USB-Stick works perfectly. So no need for the old SVN Server installation on my local server, because mercurial is file-based. But after some time I had to share my source code with other developers and here comes BitBucket into play.

The use of BitBucket was as easy as Mercurial itself. 

I've never used Git, but I was told that it was just like Mercurial. Unfortunately, this is not the case. Even the first time I used git, there were big differences and I immediately hated git. But this was not a big deal, because I could use Mercurial for my local repositories and use Git for sharing where necessary.

I know many more developers using Git that Mercurial, but I never got used to the remote and local heads for different users and the parallel branches are show differently comparing to Mercurial.

After Bitbucket broke away from Mercurial repositories and hg is no longer supported, I had to find an alternative.  After some testing, I was able to run the SCMManager on a virtual Ubuntu installation. With the online available tutorials, the installation was not complicated. Unfortunately, SSL did not work out of the box.

The SCMManager is a Java-Client and this is not so easy with the certbot for Let's Encrypt. As far as I know, you have to convert the certificates and install them into the java key store. I like the cronjob based renewal, so hand converting was not my goal.

I found a little hint, that the Nginx server could be used as a proxy server to do the https/SSL connection and leave the java installation without any changes.

Installing software on Linux with this apt-get stuff is really easy and I wish this kind of installation would also possible on Windows. One drawback often is the location or the necessary changes to config files by hand. In this case, I had to type the config file from scratch.

By adding a new A record to the nameserver I was able to run the certbot with the "--nginx" param script who not only installed the certificate but also added the necessary parameters to my handwritten config file. Well done - this kind of "service" is really nice...

With some tweaking and some reload/retries, the server was reachable over https!

Small repositories worked immediately, but large files could not be transferred and caused an hg error: "request entity too large". The reason for this problem could only be the installed proxy server because everything worked before. But also this problem could be solved by a little googling. 

So now I have my own Mercurial server. If you want to know more about this, please write this in the comments, then I will try to collect my google knowledge into another post.



Friday, October 9, 2020

How to write you own Serverside-Extention!

Serverside Extention (IIS or Apache)!

For many years I have my own webserver. In my case, this always was a Windows Server, because I want to do my serverside scripts in Delphi! Apart from that, I had no clue about a Unix/Linux Server!

Being part of the Apocalypse Coding Group opens this door and in many hours of development of our little game, I could prove, that the same ISAPI.dll could also be compiled and executed on a Ubuntu Apache Server! 

I love Delphi. 

Building a new website or web service in the past, I had to make a choice:

For the website:

1. ASP.NET (Too bad only up to D2007)
2. ISAPI.DLL

After using ISAPI.dll's for a long time I've more often used ASP.NET. The code-behind for a webpage is not locked by the server, so it was easy to update the DLL-files. This was also possible by using the well known "egg"-loader. Updating the server to a newer version suddenly prevented the loader to work and I had never the time to find the reason for this. That why I preferred ASP.NET and for the other ISAPI.dll's I always hat to start and stop my IIS-Server.

For the web service:

1. SOAP

SOAP was the way to go because it was so easy to use. Just define your Interface and use it as it is local in your application. The only drawback is the huge data that is transferred with every call. That's why I used this not in the "native" way but just to transport an encrypted, compressed binary stream.

This was the past... But as time goes by...

If your traffic is increasing and you want to do many more things with your server, there comes a time when you realize that a 2 core server with an ancient operating system is just not up to date anymore.

Virtualization is one way to go. In my case, I use Proxmox on a beefy 2 CPU, many cores,  many RAM Server with a RAID SSD drive... Nice...

This server handles two or more Windows 2019 Server and many Ubuntu Servers for small web- or microservices.

Of course, for web-services "we" are doing JSON/RESTfull web services these days. So what is needed to do all the fancy stuff? 

ISAPI.DLL's - best performance - native execution, no script interpretation. For a web-site there are two ways to go: (perhaps many more, but not for me)

1. Uses a "normal" HTML page and  AJAX background calls to get the dynamic content you need for your site.

2. Use the Webbroker implementation and deliver the content directly from your DLL. This content could also do REST-calls to get more data from the same or a different service.

Perhaps you also want some URL-Rewrite or another Server-side handling, in this case, you also need a Filter.dll.

This time I'll do it better...

With the newly installed Windows 2019-Server, I tried the simple Hello-World DLL that is included with the egg-loader. But the Server was unable to execute this little DLL. But a request to the URL just downloads the DLL. This was the same with other DLL's I just copied from the old server. (The Hello-World.DLL works without problems on the old server). After some googling. I found the "uses 32 bit - DLL's" setting. But with this setting, I got a 500 with a weird error message. Setting the error level to detailed brings more information. Spoiler: Do not google these errors! Everything you'll find is nonsense! Like: You have to disable compression or other things. Most of the ideas lead you to the Handler settings - also wrong. (of course, you have to install all the ISAPI IIS extensions, I assume you have done this). So what is the difference between the old and the new server? 

64-Bit!

OMG... Why is there a setting for 32-Bit DLL's if you have to use 64-Bit anyway? OK - Start 10.4.x, compile it to 64-Bit and try again. To bad the Loader-Version 2.0 is from 2005 and not Unicode. After some fixes - the same problems. Forgot to disable the 32-Bit setting. The Hello-Worlds.DLL brings no errors anymore but is download again with every call. The loader is "kind of " working. All log files have wide-strings. Unreadable. Again some Unicode-Fixes and recompiled the Hello-World.dll to 64-Bit...

And here we go... a nice "Hello" displayed in the browser. But the Egg-Loader is unable to update the *.run version with the *.update version. I think the IIS is still holding the DLL. (How is this possible). So I wrote an eMail to William Egge if he has any ideas... 

This error is impossible. The loader is working, the hello-world is working, a free library should also work and the IIS should have no knowledge about the site-loaded DLL. Why is the loader unable to rename the files?

OMG... Again... No rights! 

By giving the IIS-User the access rights everything is working perfectly! Of course, I wrote to William that everything is working now... Answer from Bill: "Ok Cool 😊"! Perhaps I will send him the changes or I will write a different version, that is also able to handle Filter.dll's. Not decided yet!

So here is the workflow:

1. After installing the Windows-Server and all necessary extensions.
2. Create a website!
3. Compile your WebBroker.DLL (64-Bit or matching the Server)
4. Create a directory outside of your web-page-path. (e.g. c:\API / c:\ISAPI / c:\Script)
5. Create a minimal web.config in this directory. (for debugging errorMode="Detailed")

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <handlers accessPolicy="Execute, Script" />
        <directoryBrowse enabled="false" />
        <httpErrors errorMode="Detailed" />
    </system.webServer>
</configuration>

6. Upload the DLL to this directory and rename it to WebBroker.run (or any other name).
7. Upload the loader.dll to the directory and rename it to WebBroker.dll (Same name as your DLL)
8. Add the IIS-User with access rights to this directory
9. Add an Application to your website in the IIS-Manager. Point to the ISAPI directory.

(The Name will be part of the URL) "API" -> https://www.yourdomain.com/API/WebBroker.DLL)

10. Read is not permitted and browsing is off, but anyway I would store log files outside of this directory.

With the redesign of the egg-loader, we are back in the game!

If you want to see a YouTube tutorial of this, please leave a comment. There will be a video about this as part of my Firemonkey Development Kit and the Server-Side of my JSON-Store implementation, but this is scheduled not before I've completed my #D.MVVM Framework series and more basic stuff of my FDK.

If you want a notification on new videos, please subscribe to my channel and hit the bell icon! Thank you, this helps to grow my channel!