tag:blogger.com,1999:blog-62431593946505853642024-02-08T02:14:31.661-08:00Pascal Warrior's JourneyAnonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-6243159394650585364.post-4340900166337468772013-10-14T19:34:00.000-07:002013-12-11T23:22:26.689-08:00Android Programming with Lazarus through Custom Drawn Interface<div style="text-align: justify;">OK, so you've read that it's possible to write Android applications with Lazarus and Free Pascal. You go straight to the <a href="http://wiki.lazarus.freepascal.org/Custom_Drawn_Interface/Android">wiki</a>, follow the bunch of steps there and FAIL! And then you start grumbling and doubting whether it's really possible or just a joke.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Listen up, dude. The Android support in FPC is still in development, and a pure arm-android target has just been added a couple of months ago. This target is available for those experienced enough with FPC (bootstrapping the compiler, set options for building, etc.) and not lazy to do the whole setup. Most problems come from those who don't read and do the steps thoroughly, possibly skipping important part. So if you're one of them, either change your behavior or wait until the support is available in the stable release.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I will try to explain step by step setting up FPC trunk with arm-android target support, followed by setting up Lazarus to support building Android application. Note that it's all done on Linux (Kubuntu 13.04) 32-bit, but it should work for any supported host platforms.<br /><br /></div><h3>First thing first, latest stable FPC</h3><div><br /></div><div style="text-align: justify;">FPC is a bootstrapping compiler, and it's guaranteed that latest stable version will be able to build trunk and next stable version. No guarantee for older version or between revisions of trunk, and things can be broken anytime on trunk. At this time of writing, latest stable FPC is of version 2.6.2. So grab that one if yours is not.<br /><br /></div><h3>Next, Android NDK</h3><div><br /></div><div style="text-align: justify;">For arm-android target, FPC makes use of external assembler and linker provided by <a href="http://developer.android.com/tools/sdk/ndk/index.html">Android NDK</a>. Mine is still version r8e, but looking at the changelog version 9 should work just fine. Extract it anywhere you want, we will refer to this location as<span style="font-family: inherit;"> {ndk.dir}</span>. To be sure, right under<span style="font-family: inherit;"> {ndk.dir}</span> there should be README.TXT and RELEASE.TXT.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Let's identify the tools we need:</div><div style="text-align: justify;"><ul><li>{ndk.dir}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-as (assembler)</li><li>{ndk.dir}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-ld (linker)</li></ul><div>If you want, you can change the part after /toolchains/ in case you want to use androideabi 4.6 or 4.7. Look at the corresponding directory you have in your {ndk.dir}.</div><div><br /></div><div>Open up your fpc.cfg file, by default it should contain the line:</div><div><span style="font-family: Courier New, Courier, monospace;"><br /></span></div><div><span style="font-family: Courier New, Courier, monospace;">-XP$FPCTARGET-</span></div><div><br /></div><div><span style="font-family: inherit;">This line tells the compiler to prepend any external tools called with $FPCTARGET- (note the dash), so when the compiler wants to call "as", for arm-android target, it will call "arm-android-as" instead. As you can see, the name is then inconsistent with the NDK tools name. The solution is to create symbolic links for the tools with names expected by the compiler. For hosts that don't support symbolic links (e.g. Windows), you can create a small exe wrapper for the tools</span><span style="font-family: inherit;">, or simply rename the tool. Put the symbolic links / wrappers somewhere in PATH (I put it in /usr/bin/).</span><br /><span style="font-family: inherit;"><br /></span>Ensure you do it correctly by verifying the output of <span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">ls -l `which arm-android-<toolname>`</span><span style="background-color: white; font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><span style="background-color: white;">(*nix only). It looks like this on my system (real directory replaced with {ndk.dir}):</span><br /><span style="background-color: white;"><br /></span><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">$ ls -l `which arm-android-as`</span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">lrwxrwxrwx 1 root root 120 Mar 8 2013 /usr/bin/arm-android-as -> {ndk.dir}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-as</span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">$ ls -l `which arm-android-ld`</span><br /><span style="background-color: black;"><span style="color: white; font-family: Courier New, Courier, monospace;"></span></span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">lrwxrwxrwx 1 root root 120 Mar 8 2013 /usr/bin/arm-android-ld -> {ndk.dir}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-ld</span><br /><span style="background-color: white;"><br /></span>Try executing arm-android-as and arm-android-ld in terminal or command prompt to ensure it works.<br /><br /><h3><span style="background-color: white;">Next, FPC trunk</span></h3><div><span style="background-color: white;"><br /></span></div><span style="background-color: white;">Get FPC trunk either from svn (I won't teach how to use svn, go find tutorial somewhere) or <a href="ftp://ftp.freepascal.org/pub/fpc/snapshot/trunk/source/fpc.zip">Free Pascal's FTP</a>. In case of svn, here's the address: http://svn.freepascal.org/svn/fpc/trunk</span><br /><span style="background-color: white;"><br /></span><h3><span style="background-color: white;">Build FPC for arm-android target</span></h3><div><span style="background-color: white;"><br /></span></div>Using your terminal, go to FPC trunk directory and execute the following:<br /><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">make crossall OS_TARGET=android CPU_TARGET=arm CROSSOPT='-Cp<ARM Arch> -Cf<ARM VFP>'</span><br /><span style="background-color: white;"><br /></span><span style="background-color: white;"><ARM Arch> defines the ARM architecture you want to compile for, my device is ARMv6, so I use -CpARMv6.</span><br /><span style="background-color: white;"><ARM VFP> defines the Vector Floating Point unit you want to use for floating point calculation, for at least ARMv6, </span>VFPv2 and VFPv3 are available. The default is to use soft-float, which is very slow as the calculation is performed by software. Since I seldom use floating point, soft-float is fine for me, so I don't pass any -Cf option.<br /><span style="background-color: white;"><br /></span><span style="background-color: white;">If everything goes well, it's time to install. Execute the following (still in the same FPC trunk folder):</span><br /><span style="background-color: white;"><br /></span><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">make crossinstall OS_TARGET=android CPU_TARGET=arm INSTALL_PREFIX=<directory of your choice></span><br /><span style="background-color: white;"><br /></span><span style="background-color: white;">Feel free to choose any directory you want, but ensure it fulfills the standard requirement (no space in the file path). I suggest installing to the same host FPC directory so you can easily share fpc.cfg. FPC directory structure is made such that it's possible to install cross compiler (and the respective units) in the same tree as the host compiler. The fpc driver can then be used to query which ppc[ross]XXX to call.</span><br /><span style="background-color: white;"><br /></span><span style="background-color: white;">If everything goes well, test the compiler. Execute the following:</span><br /><span style="background-color: white;"><br /></span><span style="background-color: black; color: #f3f3f3; font-family: Courier New, Courier, monospace;">fpc -Parm -Tandroid</span><br /><span style="background-color: white;"><br /></span><span style="background-color: white;">It should output something like:</span><br /><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">Free Pascal Compiler version 2.7.1 [2013/09/21] for arm</span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">Copyright (c) 1993-2013 by Florian Klaempfl and others</span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">Fatal: No source file name in command line</span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">Fatal: Compilation aborted</span><br /><span style="background-color: black;"><span style="color: white; font-family: Courier New, Courier, monospace;"></span></span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">Error: /usr/bin/ppcrossarm returned an error exitcode</span><br /><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span><h3>Next, Android SDK</h3><div><br /></div>Grab <a href="http://developer.android.com/sdk/index.html">Android SDK</a> if you haven't, r22 should be fine. We just need the SDK tools, so no need to waste time and bandwidth downloading the ADT bundle. I will refer to the SDK installation directory as {sdk.dir}. To be sure, right under {ndk.dir} there should be SDK README.TXT.<br /><br /><h3>Test AndroidLCL example, yay!</h3><div><br /></div><div>Go to your Lazarus installation directory (I will refer it as {lazarus.dir} from now on) and open examples/androidlcl/androidlcltest.lpi. Now open Project->Project Options, ensure in Target Platform OS is set to android and CPU is set to arm (or just pick the respective build mode). If upon FPC trunk building you use -Cf option, specify the same option in Other. You might need to also set it in Tools->Configure Build Lazarus dialog. Now press the Run->Build menu. If you get:</div><div><br /></div><div><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">Trying to use a unit which was compiled with a different FPU mode</span></div><div><br /></div><div>Then you don't put the -Cf option correctly. Remember you will need to put it for both your project (through Project Options dialog) and LCL (and its dependencies, through Configure Build Lazarus).</div><div><br /></div><div>If everything goes well, you will get android/libs/armeabi/liblclapp.so in the project folder.</div><div><br /></div><h3>Get Ant</h3><div><br /></div><div>Android SDK uses <a href="http://ant.apache.org/bindownload.cgi">ant build tool</a> for building apk, so you'll need to install it as well.</div><div><br /></div><h3>Build the APK</h3><div><br /></div><div>Go to android folder under androidlcl project folder, and open the build.xml. Inside, you will see 2 <span style="font-family: Courier New, Courier, monospace;">loadproperties</span> and 1 <span style="font-family: Courier New, Courier, monospace;">property</span> tags. These points to files you will need to edit to match your SDK installation. Mine is below:</div><div><br /></div><div><span style="font-family: Courier New, Courier, monospace;"><loadproperties srcFile="local.properties" /></span></div><div><span style="font-family: Courier New, Courier, monospace;"><property file="ant.properties" /></span></div><div><span style="font-family: Courier New, Courier, monospace;"><loadproperties srcFile="default.properties" /></span></div><div><br /></div><div>local.properties contains the sdk.dir which you should fill with {sdk.dir} (actual value where you install it, of course).</div><div>default.properties contains the target android API level. The complete list can be seen <a href="http://source.android.com/source/build-numbers.html#platform-code-names-versions-api-levels-and-ndk-releases">here</a>. Note that you have to install the respective SDK platform through Android SDK manager.</div><div>ant.properties contains key.store and key.alias which is required for release version of your apk. For debug version, it's not required and the apk builder utility will assign a debug key on its own.</div><div><br /></div><div>If all set, execute:</div><div><br /></div><div><span style="background-color: black; color: white; font-family: Courier New, Courier, monospace;">ant debug</span></div><div><br /></div><div>in android folder. The resulting .apk will be in android/bin folder named LCLExample-debug.apk. Install that and enjoy.</div><div><br /></div><div>From this point forward, you can make use of androidlcl structure as a template. General Java package structure and Android build system knowledge will be required to change the package name.</div><div><br /></div><span style="color: white; font-family: Courier New, Courier, monospace;">l</span><br /><span style="color: white; font-family: Courier New, Courier, monospace;">It </span></div></div>Anonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.com0tag:blogger.com,1999:blog-6243159394650585364.post-44614332338802745172013-01-18T11:48:00.000-08:002013-12-11T23:22:26.698-08:00Top Down Agile Program Development<div style="text-align: justify;">This time I would like to post something out of coding world, but rather something about software engineering.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Traditional agile program development commonly uses bottom up approach, where one builds the lowest level functionality first then goes up further. API changes is not an uncommon things to happen during agile development, which could lead to code rewriting problems at higher level(s). For example, consider the following function:</div><div style="text-align: justify;"><br /></div>function Init(var R: TSomeRecord): Boolean; <br /><br /><div style="text-align: justify;">which somehow in the middle of development gets changed to:</div><div style="text-align: justify;"><br /></div>procedure Init(var R: TSomeRecord); <br /><br /><div style="text-align: justify;">with purpose of raising an exception instead of returning boolean value to indicate successfulness of the operation.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">If this change happens in far future, then all codes using this routine must be rewritten. The problem could arise because we usually don't think how we are going to use an API, or a combination/series of them, when we design. We only think of the API as itself, standalone, away from how it will be used. Therefore, to cover the problem, we could use alternative development method, that is the top down approach.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">This method requires you to build program from its highest level first (usually user interface), then goes down further. The idea of this method is largely based on test driven development, when you write test cases first prior to implementing the test unit. The difference lies in the fact that test driven development is still bottom up in the large, yet it's top down for the unit to be implemented. This method ASSUMES the lower level API already exists, regardless of its existence. So, when you code you're thinking of using the API, possibly with other API, together to do some tasks. Because of this, the changes in the future would be minimal or even none. When something goes wrong when you test, the chance is that source of error is at lower level, which means less code to change (one place instead of several).</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">However, this method is not magical (in fact, none is), as with other methods it has disadvantages as well. You can't see immediate result of your program until the lowest level is implemented, though you can simply put "not implemented yet" output for some functionalities. Functions with deep dependencies (e.g.: requires function X which in turn requires function Y which in turn requires function Z and so on) will be the least ones visible. This method also doesn't play well with incremental approach, which is designed to be bottom up.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Final words: Choose your weapon wisely ;)</div>Anonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.com0tag:blogger.com,1999:blog-6243159394650585364.post-61897448498512535662012-12-31T10:08:00.000-08:002013-12-11T23:22:26.704-08:00Brook Framework, a new web application framework for Free Pascal<p>Recently, a new post in Lazarus forum surprised me. Somebody, OK, Silvio, announced his new web application framework for Free Pascal with over 15000 lines of code written, having integration with the great Greyhound data access framework, named <a href="http://brookframework.org">Brook Framework</a>. OK, so what's interesting from this framework? Keep reading. </p><h3>Architecture: routes, actions and methods</h3><p>Even though it's built on top of fcl-web, it doesn't make use of fcl-web architecture. Brook uses the concept of routes as commonly found in frameworks for other programming languages. For each request path (that /something/and/probably/longer thing) you want to support, you register a class (TBrookAction descendant) to handle the request path. The class itself implements method for each HTTP method the class will support (commonly GET and POST, but other methods like HEAD and PUT are also supported). This is, IMHO, a more structured yet flexible way than module-action architecture as used by fcl-web. FYI, the module-action architecture has a hardcoded request path in the form of /module/action or ?module=module-name&action=action-name. This makes the request path difficult to be made search engine friendly because you have to pass additional parameters via param1=value1&param2=value2&param3=value3 and so on. OTOH, Brook allows you to register path in almost free form (taken from <a href="http://docs.brookframework.org/BrookAction.TBrookAction.html#Register">TBrookAction.Register documentation</a>): <pre><br />* - Allows any path. Example:<br /><br />TMyAction.Register('*');<br /><br />Can be called as http://localhost/cgi-bin/cgi1, http://localhost/cgi-bin/cgi1/foo/ etc;<br /><br />/ - Adds an slash to the end of the URL if does not exist. Example:<br /><br />TMyAction.Register('/foo/');<br /><br />Can be called as http://localhost/cgi-bin/cgi1/foo or http://localhost/cgi-bin/cgi1/foo/. When called as http://localhost/cgi-bin/cgi1/foo, it will automatically redirected to http://localhost/cgi-bin/cgi1/foo/. If the pathinfo is different from /foo a 404 page is returned;<br /><br />: - Creates variables URL. Their values can be read from the property Values. Example:<br /><br />TMyAction.Register('/foo/:myvar');<br /><br />Creates the "myvar", that can be read from the property Values, e.g:<br /><br />Write(Values['myvar'].AsString);<br /><br />NOTE: Two actions can't be registered with the same pattern except when they are called by means of different HTTP methods.<br /></pre></p><h3>Data access integration</h3><p>Actions could optionally have direct data access. To do that, the action must descend from TBrookDBAction instead of TBrookAction. The action could then read/write data from/to database (or something else) while serving a request. Actually, my experience in a good MVC structure teaches me not to tie the request handler and the data persistence layer. But since this is optional in Brook, I can live with it. Besides, it can be a good thing for quick development. </p><h3>Enough talk...</h3><p>So let's get started, we'll be using Lazarus for easiness. <ol><li>Download brook <a href="http://brookframework.org/download/release.bf">here</a></li><li>Extract it somewhere, open packages/brookex.lpk with Lazarus and install, this will register entries for easily creating new brook projects in Project->New Project menu</li><li>Next, pick up one of the available data access backends. Since I'm currently playing a lot with Greyhound, I pick up brookgreyhoundrt.lpk. You may pick something else if you like such as ZEOS or ADS backend. <li>After Lazarus restart, pick Project->New Project menu, you'll see 2 new entries named "Simple CGI Application" and "Full CGI / FastCGI Application". Pick up the latter as it provides more features</li><li>A dialog with form will appear, the fields are intuitive so I guess I don't have to explain. Just fill the form and press Next</li><li>Another dialog will appear. Here you can set the actions you want and their respective path, optionally setting which one will be the default (if no specific path given the request). There's a button "Patterns help" that redirects to TBrookAction.Register documentation exactly like in the previous section. When you're done, press Next</li><li>Congratulations! Simply skip (doh)</li><li>A project will be created with one unit per action you register, and a bunch of predefined files: 404.html, 500.html, Brokers.pas and the .lpr. The most important file is Brokers.pas. This unit acts as a central configuration settings. So, whatever configuration you need, set it here. You'll see that it already registers the 404 and 500 page. For FastCGI application, you can set port here by using: <pre class="delphi" name="code"><br />TBrookFCGIApplication(BrookApp.Instance).Port := {Your port number here};<br /></pre>Don't forget to add BrookApplication and BrookFCLFCGIBroker to the uses clause</li><li>Now open up an action unit and you'll see the Get method is already overriden with a default content. You can edit that later to produce html or whatever output you want. For now, we just want to test that it works</li><li>Build the project and run (in case of FastCGI) or copy to your webserver's cgi directory (in case of CGI)</li><li>Now go to your browser and type the url to your application, I personally use FastCGI with Nginx on port 8080, and if my action is /index/, I'll type in my browser: http://localhost:8080/index/</li><li>If you see your output, then you've managed to make it work. Feel free to improve</li></p><p>Pascal for web? Why not? ;)</p> Anonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.com0tag:blogger.com,1999:blog-6243159394650585364.post-39105586861604251532012-06-24T09:25:00.000-07:002013-12-11T23:22:26.710-08:00Encryption / Decryption and Asynchronous Socket Programming<br />Back again, it's been a long time since my last post due to lack of free time and laziness :p<br /><br />Recently, I've got some posts in Lazarus / Free Pascal forums asking for some incompletely documented features, namely the (en|de)cryption unit (blowfish) and asynchronous socket (from fcl-net). Free Pascal is shipped with huge powerful libraries which are mostly, unfortunately, undocumented. Through this post, I hope I can help document it a bit through examples (I'm still lazy for real documentation commit :p). Let's start, shall we?<br /><br /><h2> Blowfish, the cryptography unit</h2>current documentation: <a href="http://www.freepascal.org/docs-html/fcl/blowfish/index.html">http://www.freepascal.org/docs-html/fcl/blowfish/index.html</a><br /><br />This unit implements encryption / decryption classes with keys, and is able to apply it on any TStream descendant. For easiness, we'll use TStringStream for the example. On to the code:<br /><br /><pre class="delphi" name="code">{$mode objfpc}{$H+}<br /><br />uses<br /> Classes,<br /> BlowFish;<br /><br />var<br /> en: TBlowFishEncryptStream;<br /> de: TBlowFishDeCryptStream;<br /> s1,s2: TStringStream;<br /> key,value,temp: String;<br />begin<br /> { 1 }<br /> key := 'testkey';<br /> value := 'this is a string';<br /> { 2 }<br /> s1 := TStringStream.Create('');<br /> en := TBlowFishEncryptStream.Create(key,s1);<br /> { 3 }<br /> en.WriteAnsiString(value);<br /> en.Free;<br /> WriteLn('encrypted: ' + s1.DataString);<br /> { 4 }<br /> s2 := TStringStream.Create(s1.DataString);<br /> s1.Free;<br /> { 5 }<br /> de := TBlowFishDeCryptStream.Create(key,s2);<br /> { 6 }<br /> temp := de.ReadAnsiString;<br /> WriteLn('decrypted: ' + temp);<br /> <br /> de.Free;<br /> s2.Free;<br />end.</pre><br />Explanations per curly brackets:<br /><br /><ol><li>First, we prepare the key (key) and data to be encrypted (value)</li><li>Next, we create a TBlowFishEncryptStream instance, providing the key and stream to write the encrypted data into (s1)</li><li>Now we write the unencrypted data. For testing, we output the encrypted data. You'll see that it would be a bunch of weird bytes</li><li>Next, we will try to decrypt the data back to its original form. First, we create another TStringStream, this time we give the encrypted data as the stream data</li><li>Then we create a TBlowFishDeCryptStream instance, providing the key that was used to encrypt the data and the stream from which the encrypted data would be read</li><li>Next, read the data and output it. You'll see it's the original 'this is a string'</li></ol><div>So easy, huh? On to the next one.<br /><br /></div><h2> fcl-net, the undocumented treasure</h2><div>current documentation: err.. none</div><div><br />This package offers a lot of networking things: asychronous socket, dns resolver, HTTP servlet, socket streams, etc. We would concentrate only on the asychronous socket (and implicitly, socket streams). At first glance, it looks uneasy to use. I have to dig in the sources to see how it works and guess how to use it. We will implement a server with multiple client support. To stay focus, the client will only connect, send a 'hello' message, then disconnects. The server would display notification for an incoming connection, the message sent by the client, and when the client disconnects. The server can only be terminated with Ctrl+C. Jump in to the server code:<br /><br /></div><pre class="delphi" name="code">{$mode objfpc}{$H+}<br /><br />uses<br /> { 1 }<br /> {$ifdef unix}cthreads,{$endif}<br /> Classes,SysUtils,Sockets,fpAsync,fpSock;<br /><br />type<br /> { 2 }<br /> TClientHandlerThread = class(TThread)<br /> private<br /> FClientStream: TSocketStream;<br /> public<br /> constructor Create(AClientStream: TSocketStream);<br /> procedure Execute; override;<br /> end;<br /> { 3 }<br /> TTestServer = class(TTCPServer)<br /> private<br /> procedure TestOnConnect(Sender: TConnectionBasedSocket; AStream: TSocketStream);<br /> public<br /> constructor Create(AOwner: TComponent); override;<br /> end;<br />{ 4 }<br />function AddrToString(Addr: TSockAddr): String;<br />begin<br /> Result := NetAddrToStr(Addr.sin_addr) + ':' + IntToStr(Addr.sin_port);<br />end;<br /><br />{ TClientHandlerThread }<br />{ 5 }<br />constructor TClientHandlerThread.Create(AClientStream: TSocketStream);<br />begin<br /> inherited Create(false);<br /> FreeOnTerminate := true;<br /> FClientStream := AClientStream;<br />end;<br />{ 6 }<br />procedure TClientHandlerThread.Execute;<br />var<br /> Msg : String;<br /> Done: Boolean;<br />begin<br /> Done := false;<br /> repeat<br /> try<br /> Msg := FClientStream.ReadAnsiString;<br /> WriteLn(AddrToString(FClientStream.PeerAddress) + ': ' + Msg);<br /> except<br /> on e: EStreamError do begin<br /> Done := true;<br /> end;<br /> end;<br /> until Done;<br /> WriteLn(AddrToString(FClientStream.PeerAddress) + ' disconnected');<br />end;<br /><br />{ TTestServer }<br />{ 7 }<br />procedure TTestServer.TestOnConnect(Sender: TConnectionBasedSocket; AStream: TSocketStream);<br />begin<br /> WriteLn('Incoming connection from ' + AddrToString(AStream.PeerAddress));<br /> TClientHandlerThread.Create(AStream);<br />end;<br />{ 8 }<br />constructor TTestServer.Create(AOwner: TComponent);<br />begin<br /> inherited;<br /> OnConnect := @TestOnConnect;<br />end;<br /><br />{ main }<br />{ 9 }<br />var<br /> ServerEventLoop: TEventLoop;<br />begin<br /> ServerEventLoop := TEventLoop.Create;<br /> with TTestServer.Create(nil) do begin<br /> EventLoop := ServerEventLoop;<br /> Port := 12000;<br /> WriteLn('Serving...');<br /> Active := true;<br /> EventLoop.Run;<br /> end;<br /> ServerEventLoop.Free;<br />end.<br /></pre><br />It's a bit long, so take a breath:<br /><br /><ol><li>We will need each client to be handled in its own thread, so we need cthreads unit for *nix OSes</li><li>The client handler thread, it would work on the given client socket stream</li><li>The server, we will create an override constructor to hook when a client connects</li><li>Helper routine to get ip:port as string</li><li>Overriden constructor for the thread, will call the inherited constructor (with false argument to indicate the thread shouldn't be in suspended state), setting the object to free itself whenever the execution has finished, and assign the socket stream to a private attribute</li><li>The core of the thread. Will try to read what the client sends and output it in the server log until the client disconnects</li><li>OnConnect handler, prints notification message whenever a client connects and create a handler thread for it</li><li>Overriden constructor for the server, assigns the OnConnect handler</li><li>Main program. Create event loop object for the server, creates the server, assigning event loop and port to listen, and ready to accept connections...</li></ol><div>Phew, go on to the client. This time it's simpler:</div><div><br /></div><pre class="delphi" name="code">{$mode objfpc}{$H+}<br /><br />uses<br /> Classes,SysUtils,Sockets,fpAsync,fpSock;<br /><br />var<br /> ClientEventLoop: TEventLoop;<br />begin<br /> ClientEventLoop := TEventLoop.Create;<br /> with TTCPClient.Create(nil) do begin<br /> EventLoop := ClientEventLoop;<br /> Host := '127.0.0.1';<br /> Port := 12000;<br /> Active := true;<br /> EventLoop.Run;<br /> Stream.WriteAnsiString('Hello');<br /> Active := false;<br /> end;<br /> ClientEventLoop.Free;<br />end.<br /></pre><br />Not numbered since it's only a single main code block. First it creates event loop for the client, create the client, assigning event loop, host and port of the server to connect, activate the connection, send a 'Hello' message to the server and disconnects. Clear enough, eh?<br /><br />OK, that's all for now. Got any questions? Just ask :)Anonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.com0tag:blogger.com,1999:blog-6243159394650585364.post-11429515524452239972011-01-14T04:15:00.000-08:002013-12-11T23:22:26.716-08:00Big mistake: wrong process model<p>This is a lesson learn for application developer. Remember my last post about compiler with LLVM backend? It doesn't end happily. I didn't manage to connect all the components, even some of them are either incomplete or not implemented at all. I pass the trial session badly. Even though I managed to graduate, but the ending is not good.</p><p>So, after the trial session, here comes the revision session. I crazily decided to rewrite the compiler FROM SCRATCH. But this time, I use feature based iterative incremental process model (previously component based iterative incremental). First, only single expression is allowed. Parser OK, AST OK, semantic checking OK. Next, assignment statement. Parser OK, AST OK, semantic checking OK. Next, compound assignment statement, return statement, method, class and so on. Ended with code generator. Guess what? Something that I didn't manage to finish in 3 months is finished in 1 week!</p><p>If only I realize this faster... maybe I'll get A (well, I finally get A- which is not too bad).</p>Anonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.com0tag:blogger.com,1999:blog-6243159394650585364.post-60914797971667024832010-12-24T00:11:00.000-08:002013-12-11T23:22:26.722-08:00LLVM IR Builder in Object Pascal<p>I'm about to graduate from my university (January 2011 if there's no more problems), and as a final assigment (though it's optional in my faculty, but it would be a great experience and honor to have one) I choose to implement a compiler. One of our labs, Formal Method in Software Engineering (or simple FMSE), is the lab where my supervisor gets involved. Therefore, the compiler I'm writing would probably based on a project they're (or have been) working on. Yep, I was given <a href="http://fmse.cs.ui.ac.id/?open=lingusql/index">LinguSQL</a> language to implement.</p><p>Previously, the language had a compiler, that generates Java code (bad choice IMO) which is then compiled by a Java compiler. The problem with this approach, as in normal Java application, is the HUGE runtime environment that must be distributed if someone wants to use the application. Furthermore, since Java uses interpreted bytecode (don't count GCJ, I even believe less than 10 persons in my campus know that thing exists), the performance is at maximum only 1/3 of native binaries (someone in OSDEV forum ever said). Last but not least, the execution isn't trivial. One must type "java xxx" in order to execute the program.<p></p>As a native application (deve)lov(|p)er (read: developer and lover), I decided to implement a compiler for this language that generates native binaries. However, I don't have enough experience in generating native binaries (or at least native assembly). So, remembering an option, I asked my supervisor what if the compiler generates LLVM assembly language (also called LLVM Intermediate Representation or IR)? Do you what he said? "What is LLVM?" (doh). OK, so after bla bla bla, he accepted my choice. The advantages of generating LLVM assembly instead of native assembly are:<br /><ol><li>A LOT of optimizations for free</li><li>It can be compiled to native assembly for MANY platforms</li><li>Easy integration with existing libraries</li></ol>Despite those advantages, there are also disadvantages:<br /><ol><li>It uses SSA format</li><li>Written in C++, more specifically, it officially only supports G++! (there are some hacks to use MSVC but... still it's not official)</li><li>Most important one: it doesn't have Object Pascal frontend</li></ol>The SSA format is not actually a disadvantage, but it's just a little harder to generate code for. But the last one is really a show stopper... or a challenge depending on how you look at it :)<br /></p><p>So the work begins. I continue the previous research, the previous Java based compiler that generates Java code uses JavaCC, followed by JavaCUP, and finally UUAG. The first two are parser generators, with some differences, mainly JavaCC generates LL parsers, while JavaCUP generates LALR one. Both are BAD. I always find parser generators are bad since we have no idea whether it's correct or not, and the grammar can't be deduced from the code (except for recursive descent parser generators like Coco/R). The last one is a Haskell based product, which actually runs like recursive descent parser, only in functional languages they're called parser combinators. The last one is quite good, with one important problem: when a parsing error happens, the parser tries to find all possible corrections, therefore slows down the parsing and eats resources. This behavior can't be customized easily and that's what makes me writing the whole thing from scratch using classic approach: a true recursive descent parser. This is the best parser I've ever learned, since it's the most flexible one (there are tons of way to handle parsing error and that's totally up to you, with many methods possibly combined or used specifically for certain productions) and still shows the grammar in its code.</p><p>Come to the code generation part, the problem I stated above must be covered. I create my own LLVM IR Builder to generate LLVM assembly language. Due to the SSA structure, it's a bit difficult, but I managed to create it quite successful with beautiful modular architecture. It can now generate modules consisting of functions and global variables, where each functions can have local variables, labels (for branch and loop), arithmetic instructions, memory instructions, etc. It's not yet complete, but already capable of generating simple programs. I'll put it in my bitbucket account when I think it's quite production ready.</p><p>Wants some code? OK:<br /><pre name="code" class="delphi">program llvmirbuildertest;<br /><br />{$mode objfpc}{$H+}<br /><br />uses<br /> llvmirbuilder;<br /><br />var<br /> x,y,l,s,a,b: TLLVMSymbol;<br /> c: TLLVMConstant;<br /> cl: TLLVMCallInstruction;<br />begin<br /> x := TLLVMSymbol.Create('x',lltInteger,true);<br /> y := TLLVMSymbol.Create('y',lltInteger);<br /> c := TLLVMConstant.Create('255',lltInteger);<br /> l := TLLVMLoadInstruction.Create('tmp',lltInteger,x);<br /> s := TLLVMStoreInstruction.Create('tmp',lltInteger,y,x);<br /> cl:= TLLVMCallInstruction.Create('func',lltInteger);<br /> a := TLLVMAddInstruction.Create('a',lltInteger,x,c);<br /> b := TLLVMSubInstruction.Create('b',lltInteger,c,y);<br /> WriteLn(l.GenerateCode);<br /> WriteLn(s.GenerateCode);<br /> WriteLn(cl.GenerateCode);<br /> WriteLn(a.GenerateCode);<br /> WriteLn(b.GenerateCode);<br /> a.Free;<br /> b.Free;<br /> cl.Free;<br /> s.Free;<br /> l.Free;<br /> c.Free;<br /> y.Free;<br /> x.Free;<br />end.<br /></pre>and the generated LLVM IR:<br /><pre name="code">%tmp = load i32 * @x<br />store i32 %y, i32 * @x<br />call i32 @func()<br />%a = add i32 @x, 255<br />%b = sub i32 255, %y<br /></pre>Note that it's a partial code, so compiling this with llvm-as would absolutely produce an error.<br /></p>Anonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.com0tag:blogger.com,1999:blog-6243159394650585364.post-86837093769673577712010-12-18T07:05:00.000-08:002013-12-11T23:22:26.728-08:00Using Google Maps service in a Pascal based Web Application<p>My final team assignment of Information Retrieval (IR) class allows us to create any kind of IR system, and we decided to create a food ingredients and restaurant search system. It displays ingredients for a chosen food and then displays 10 top restaurants that provide the food. For the first one, it's simply a database approach. But the latter is a Geographic IR (GIR). Instead of creating it from scratch (which could take probably a whole life), we decided to use google maps service.</p><p>Due to the requirement that the system must be accessible from web, it's splitted into two parts: client and server. The client part is coded by my friend and the server part is mine. The client part is coded with ExtJS and we use JSON to communicate between the client and server. Since the server is a Service Oriented Application (SOA), it's no problem what language it's written in. And since I'm a Pascal geek (you can call me maniac if you want), I choose to write it in Pascal.</p><p>To create the server, I use fpWeb components available from standard Lazarus distribution. The server itself connects to Google Maps service to look for the coordinates. Hmm... how can the server do this? Luckily, there are two powerful networking components and libraries for Pascal, namely lNet and Synapse. I used to lNet due to its component based approach, however after struggling a bit, lNet seems to leak some features I need (esp. proxy support because my campus' internet access is protected by that). Then another Pascalian told me that it's very easy to do with Synapse. And so it goes:</p><pre name="code" class="delphi">with THTTPSend.Create do<br /> try<br /> ProxyHost := ConfigFile.ReadString('proxy','host','');<br /> ProxyPort := ConfigFile.ReadString('proxy','port','');<br /> URL := Format('http://maps.google.com/maps/geo?q=%s&output=json&oe=utf8&sensor=true' +<br /> '&key=%s',[QueryStr,GoogleAPIKey]);<br /> if not HTTPMethod('GET',URL) then begin<br /> AResponse.Content := Format('Error %d: %s',[ResultCode,ResultString]);<br /> end else begin<br /> AResponse.Contents.LoadFromStream(Document);<br /> end;<br /> AppendToLog(Headers.Text + AResponse.Content);<br /> finally<br /> Free;<br /> end;<br /> Handled := true;<br /></pre><p>The server uses TIniFile (as ConfigFile in above code) to store and retrieve proxy information so it can be adjusted without recompilation. QueryStr is the string that we want Google Maps to look for and GoogleAPIKey is the... err.. API key to access Google Maps. FYI, I'm using version 2 of the API which still requires a key, version 3 doesn't need it anymore.</p><p>It doesn't yet show anything useful, only the original JSON from Google Maps. Later, this JSON must be processed and merged with database result, and then sent to the client to be processed further. But after all... it's done in Pascal :)</p>Anonymoushttp://www.blogger.com/profile/15670561125894701280noreply@blogger.com0