Today is a good day to code

Adding Machine Learning to Nc3 Bb4 Chess

Posted: September 29th, 2011 | Author: | Filed under: artificial intelligence, chess, JavaScript, nc3bb4, Programming | Tags: , , , , | No Comments »

While the garbochess engine is plenty strong used in the Nc3 Bb4 Chromebook chess game, I thought it would be interesting to look at adjusting the weighting mechanism by sucessful and unsuccessful outcomes.

The first thing I had to look at was how garbochess weights potential moves.  This took me into the super interesting world of bitboards.  A quick aside,  I have been working on mapreduce for the past few weeks, so looking at early methods of dealing with big data ( chess has an estimated ~ 10120 ) legal moves, in order to successfully evaluate all of the possible moves for a given position, plus all of the possible counters, weight them and choose the best possible move given criteria certainly qualifies as big data.

Interestingly, the approach wasn’t the hadoop approach, the hardware to use such brute force methods wasn’t available, instead early chess programmers tried to filter out undesirable moves, or obvious bad moves, moves that had no clear advantage, etc… What they ended up with was a pretty manageable set of moves for a circa 2011 computer.

The way garbochess considers moves, it looks at mobility for a given piece, control of the center, if a capture is possible, what the point differential for a trade would be, etc… and assigns a score for each possible legal move, it then runs through it repeatedly re-scoring the set relative to the available moves, removing the lowest scored moves, etc… eventually coming up with the best possible move.  What I wanted it to consider, was given that and the specific weights, mobility vs actual point value for a given piece, to use a markov chain for reinforcement learning to describe the entire process of a game and then rate each move with a weight enhancement upon endgame moves as being more important.  Every time the machine takes an action that leads to a success, the heavier the bias on the scoring for a given action.  Failure doesn’t automatically nullify the learning, but it definitely has an effect.

Where I got was a rudimentary implementation of this, as a bunch of housekeeping chores popped up, for example, as this is JavaScript, and all I really have is HTML5 storage, how do I store all of the moves while keeping the system responsive, no O(nn) or O(n2) lookups, what I wanted was to keep it O(n). Obviously that called for a HashMap of a sort, but the serialization / deserialization plus the key system were a challenge.  I didn’t want for it to cause too much overhead for the map / scoring system, as the bit twiddling is already pretty efficient, so I did the best that I could using the FEN + PGN.  The FEN is the state for the markov chain, since one could have a given PGN in many situations, and the weighting system could never be applied against the gravity of the situation.

I need to do more work on weighting changes based on how in trouble the machine is, whether they have an advantage or not, etc… But for a start with machine learning in chess, I think it works.


Why My Faith in HTML5 Has Been Reinstated ( Or How I Learned *again* to Love JavaScript )

Posted: July 13th, 2010 | Author: | Filed under: CSS, JavaScript, Programming | Tags: , , | 1 Comment »

Over the long weekend, I was lamenting over how many times I had to write the same routines in different languages… Objective-C, Java, PHP, etc… I realized that I have, and would have wasted tons of time writing native code, and how, really most of the functionality of the application can be handled with various features of HTML 5.  Originally I had been against this, but now that the iPhone has finally caught up and has a reasonable processor, I think that the HTML5 experience can be nearly as good as native.

The funny thing is that much of what is driving my decision is the desire to have my applications have data and interfaces that are available everywhere, mobile, web, desktop, etc… aka, the original promise of the web.  Using local caching, the JavaScript key/value store, and the database will help to allow me to provide a compelling disconnected use experience.  The code that I write will be useful across all of the platforms that I use.  The one caveat that I am making is that I need to focus on one browser, or approach, and for that WebKit based browsers seem to be the logical choice.

Now I realize that not all of my application concepts will be possible with HTML5 and JavaScript, however this recent thought experiment I realized that most of the features that I would have normally insisted needed to be done natively can be done with HTML5.  The biggest issue that I have run across is the 5 MB limit on database sizes in Mobile Safari.  I know it was there in iOS 3.x, I don’t think they have lifted this in the current OS.  The other issue is the forced UTF-16 encoding of characters.  While I understand this technically, it makes it difficult to store data larger than 2.5 MB on a device in the SQLite storage available to JavaScript.  The approach taken in desktop Safari, where you can ask the user to increase the available size if your database creation fails is a much better approach.

Another interesting pattern that I see emerging is that of utilizing HTML5 as the UI tier, and establishing the business logic and control structure behind a HTTP server that would expose additional native functionality to the HTML5 app.  The benefit here is that your local server implementation could match the remote server implementation, such that your client APIs could remain consistent.  This seems to me to be the best architecture for minimizing the work involved with porting solutions across platforms.  I absolutely love Cocoa and Objective-C, I enjoy the concepts behind the Android APIs, while despising Java’s syntax, and I think that .net is pretty cool as well, however when it comes to getting applications deployed to the maximum number of users in the leanest manner possible, I think it makes sense to leverage the web heavily up front, and then backfill the native implementations as necessary.


Mides 1.7 New Features and Changes

Posted: January 25th, 2010 | Author: | Filed under: Apple, iPhone, JavaScript, mides, PHP, Programming | Tags: , , , , , | 2 Comments »

I wrote Mides originally to help me to write web applications when I am on the go.  A huge part of web application development is JavaScript.  The iPhone / iPod is an awesome device for heavy client JavaScript apps.  So as a result, I added JSLint in Mides 1.7 to make debugging JavaScript easier.

The main problem with the developer setting in Mobile Safari is that it is inaccessible to other applications.  Since one of the main purposes of Mides is to enable development with either no, or an unreliable internet connection, it wouldn’t be possible for Mides’ internal HTTP server to run and serve the mobile safari application with content.  This is the entire reason I wrote an HTTP server for Mides, so that JavaScript XHRs would work correctly for testing.

What I have done to help out with JavaScript debugging is to modify Douglas Crockford’s JSLint library slightly to make it work on the iPhone.  It helps out with outright errors, but also with many excellent tips for writing safe and readable JavaScript applications.  You can see the errors and optimizations by tapping on the burst and exclamation point icon when it appears over your JavaScript or HTML.  This feature is optional and can be disabled in the iPhone settings.

Another issue I wanted to address with a new feature is that I always forget the argument, or the exact PHP method call that I want to use, especially around MySQL.  I already had the documentation in there, but since it is a full-text search, it tends to take a while.  So I added a new feature that allows you to look up just the method signature, that is the method name and the arguments to the method.  I didn’t want to put a button in there for this, it just didn’t seem right.  I tried for a while to come up with something usable, and I think I have figured out something that works.  You just need to twist the phone to the right ( or left ) to do the code-completion on the method.  If the text before the cursor matches one or more PHP method signatures, then it will add that value in context, in line into your code with the argument types.  If it matches more than one, it will display a modal dialog that will allow you to choose from the top 5 PHP methods that match what you have typed.

One fix that a customer asked for on getsatisfaction.com/mides was that I make tabs parse properly.  I also added that in Mides 1.7, now your tabs will be properly displayed.  To create a tab, just space 5 chars into the document.

I am adding features both at the request of customers on the burgeoning community on getsatisfaction, as well as through my own usage of the product.  I probably won’t implement all of them, but please keep the suggestions coming.  They help tremendously.  Some of them are really tough to implement, but if they make it more usable I’m all for it.

One of the main issues around Mides is moving files onto and off of the phone, Apple hasn’t made it easy, and FTP is not the best solution, it is a nightmare to support, and difficult for users to set up.  I thought about having a small application that you could install on your Mac and PC that would make it much easier to transfer files with, but this didn’t seem like the best solution either.  I am actively thinking through better ways, but nothing so far has really stuck.

At any rate, I am constantly trying to make Mides more useful, I know it has been rough, but I’m glad to see that some of you are starting to get real use out of Mides.  I hope to keep making it better and eventually to rival and in some ways improve upon the desktop coding experience.


Which JavaScript Framework is the Fastest

Posted: April 8th, 2009 | Author: | Filed under: JavaScript, Programming | Tags: , , , , , | No Comments »

I have been wondering for a while which JavaScript framework is the fastest. I was thinking about writing some sort of test to try to determine which one had the best speed, but I have found one that seems to work. Mootools SlickSpeed Test is a good start.

It seems to focus on DOM manipulation / access / iteration speed, rather than testing the functionality built into the frameworks. I suppose that it would be tough since each framework offers different things. When I ran the test, Prototype 1.6.0.2 was the slowest, YUI 2.5.2 was the next slowest, MooTools 1.2 was next up from the bottom, JQuery 1.2.6 was the second fastest, and Dojo 1.1.1 was the fastest by a wide margin in Safari 4 beta, albeit with some errors.

In Google chrome 2.n beta, the results were as follows:

  1. JQuery 1.2.6
  2. MooTools 1.2
  3. Dojo 1.1.1
  4. YUI 2.5.2
  5. Prototype 1.6.0.2

In Firefox 3.0.6

  1. MooTools 1.2
  2. JQuery 1.2.6
  3. Prototype 1.6.0.2
  4. Dojo 1.1.1
  5. YUI 2.5.2

In IE 8 ( Wow IE 8 is slow )

  1. Dojo 1.1.1 ( many errors disqualified )
  2. JQuery 1.2.6
  3. YUI 2.5.2 ( a few errors )
  4. MooTools 1.2
  5. Prototype 1.6.0.2

iPhone Safari ( DNF / Could not run / Simulator)

  1. JQuery 1.2.6
  2. MooTools 1.2
  3. Dojo 1.1.1
  4. Prototype 1.6.0.2
  5. YUI 2.5.2

Android Browser

  1. JQuery 1.2.6
  2. MooTools 1.2
  3. Dojo 1.1.1
  4. Prototype 1.6.0.2
  5. YUI 2.5.2 ( Big Suprise )

What is interesting about these tests is that in general it seems that you should use JQuery if your development pattern involves heavy selector use. I still prefer Prototype because of the programming features that I get with it, even if the selector part is slow. IE 8 breaks a lot of the frameworks. Prototype and JQuery hold up the best it seems. I haven’t really looked at MooTools however.

On mobile devices, you should think long and hard about using any framework that involves added overhead since the devices are really slow. It seems that Dojo supports the built in Safari functions for dom navigation or something. It was wicked fast in Safari 4, but had a few errors. Overall JQuery is probably the best. I guess I’ll have to take a look at it, though reluctantly. I still need to write a test to check iterator performance though.


Safari 4 – Worker Threads… JavaScript Domination

Posted: February 24th, 2009 | Author: | Filed under: Apple, Companies, Google, JavaScript, Microsoft, Programming | Tags: , , , , , , | 1 Comment »

I do hope you will pardon the hyperbole a bit, but If someone had told me a few months ago that we would have JavaScript threading, which I have been begging for for years, built into the HTML standard.  I would have thought they were crazy.  Now we have a situation where Safari 4, Firefox 3.1, Chrome ( Gears ), and IE 8 ( all in beta ) support it.

Lets look into my crystal ball for a minute.  We have a situation where browser based apps are becoming more and more capable all the time.  Where arguably the most efficient method for developing against mobile devices is to use web technologies, and where we have an insanely awesome JavaScript engine available for general use in any programming system in Chrome.  Looking down the line, I can see that JavaScript will be the primary development language once we start seeing implementations for HTML 5 Web Sockets.  It may be there, I just haven’t checked yet…

If you have Safari 4, or the webkit nightlies, you’ve got to check out this link:

JavaScript Ray Tracer

The speed of JavaScript as an interpreted language is up there with any of the others, in fact, Firefox 3.1, Chrome, and Safari 4 are wicked fast.  Soon, we may not need desktop apps at all, and Microsoft’s bungled ActiveX dream may just come to pass.  What an exciting time to be a developer!


Safari and Standards Complicance

Posted: December 31st, 1969 | Author: | Filed under: JavaScript, Programming, Uncategorized | Tags: , , | No Comments »

Safari and Standards Compliance

Picture of Irv Owens Web DeveloperApple with Safari 2.0 has taken a major step toward standards compliance and largely are taking a leadership role in this area with its outstanding support for the Java runtime. I have heard some griping about Apple using KHTML, the default rendering technology behind the Konqueror browser for KDE, for a base, then running away with the open source once they have figured it out and not giving it back to the OSS community.

While I am extremely happy that Apple has made their browser Acid2 compliant, and they may have one of the fastest CSS rendering engines around built into the AppleWebCore. It is pretty upsetting that they would not share these advances with the developers working on KHTML so that it could also pass the Acid2 test. I can understand that some things you want to keep close to your vest for security reasons, but I can hardly believe that changes you have made to the way pages render in a browser could compromise your system integrity. This appears to be a situation in which Apple wants to be the most standards compliant platform on the market. This would be fantastic from a business standpoint since many in the scientific and mathematics communities would probably prefer to use technology that adhered to standards so as to better communicate information between offices, regions, and countries. I can understand that Apple wants to distinguish its platform from others, and I love the fact they are using standards compliance to do this, however I feel that it is to break the spirit of open source / corporate collaboration not to give something back to the KHTML community.

Speaking of Safari, I noticed a bug recently while writing some javascript for it. I have a javascript that sets the tabindex for a number of input fields, and it works properly, however in Safari it persists in scrolling the real browser scrollbar instead of the div, overflow:auto, element's scrollbar. I had noticed this way back in Safari 1.2 where if you put a flash item within a scrollable div, it would take the flash element and while scrolling lay it on top of all your other content, even if it was above or below the div. All other browsers, even IE 6, handle this properly, scrolling the div with the tabbing. This is a pretty big bug if they want to promote standards compliant web development and accessability. I'd like to see this fixed in Mac OS X 10.4.1, but after browsing the message boards elsewhere, I'd say they already have their hands full, so I am not supremely hopeful.

Microsoft is promising that its IE7 browser will be standards compliant, but just how standards compliant is really the question. I think that Microsoft has learned the error of its proprietary ways. Sure it will continue to bundle its software with everything anyone buys from them, but I don't think they will continue to cripple other products to make theirs look better. They seem to have given up on their own version of DHTML and are happy with XHTML. I noticed that their primary page even validates now. I think that it makes sense for Microsoft to go the standards route also, and with no shortage of developer feedback, they have almost no excuse not to.


Internet Explorer 7 Won’t Make the Grade on Acid

Posted: December 31st, 1969 | Author: | Filed under: JavaScript, Programming, Uncategorized | Tags: , , | No Comments »

Internet Explorer 7 Won't Make the Grade on Acid

Picture of Irv Owens Web DeveloperAs the market leader and pace-setter as far as which technologies make the cut for the web Microsoft has a responsiblity to create the most standards compliant browser possible, even at the risk of breaking legacy sites built specifically for IE. Microsoft has always wanted developers to use it's unusual flavor of IE. Whether it is by building extra padding into block level elements regardless of how the css padding attribute is used, or allowing oddities like allowing the use of the color attribute on TR table elements, developers have always had to consider the quirks of IE when building anything for deployment over the web.

I'm sure that IE 7 will be much improved over IE 6 as far as standards compliance is concerned, and some of those oddities I truly enjoy, like being able to give a TR an ID attribute and specifying a header style for my tables in a stylesheet, but at the same time, if we don't have web standards we'll devolve into fragmented development languages like it was 1995 all over again. IE 6 actually had excellent standards compliance when it came out, but times have changed and there are some advanced features like page-break-after that I'd love to use more widely. Part of the reason I love to build intranet applications for Mac only shops is that I know they will be using Safari 2.0 which is an excellent browser based on the open source Konqueror browser bundled with many Linux distros. It supports most if not all CSS 2 tags, and should pass the Acid2 test with ease. Also, by developing to XHTML 1.0 Strict I know that my site will degrade gracefully on everything from mobile devices to old 3.0 browsers. Using ECMAScript also keeps most backward compatability and allows developers to create reliable JavaScripts that will work across all compliant browsers in the same fashion.

I agree with Hakon Lie that Microsoft should really take more time and make sure they nail this one, not just for right now, but for the future since we all know they won't release another web browser perhaps forever since they are convinced that Avalon will change the face of web applications and render the web browser superfluous. We've heard that one before, remember Active X? I hope that everyone calls on Microsoft to work to get IE 7 to pass the Acid2 test, not just so that it will support some bizarre standard that is going to make all our lives harder, but so that developers can be sure that applications they develop today will still look and work the same five years from now. C'mon Microsoft please?

Next Explorer to fail Acid Test – CNET


Configuring ColdFusion MX 7 and Apache

Posted: December 31st, 1969 | Author: | Filed under: JavaScript, Programming, Uncategorized | Tags: , , | No Comments »

Configuring ColdFusion MX 7 and Apache

Picture of Irv Owens Web DeveloperAnother issue I kept coming across during my configuration of the XServe G5's Apache and JRun4 was that the virtual hosts didn't seem to be resolving. The same site appeared to collect all the hits. After several hours last night troubleshooting, I finally found the culprit.

When the JRun / Apache bridge is configured, a small module is built and plugged into Apache that allows it to process ColdFusion templates from within its default web root. This functionality is great, it allows a user to serve up .jsp, .php, and .cfm files from the same folder. A single modification is needed to JRun to allow web users to get to your files without having to add /cfusion to the end of their URL request. In JRun there is a setting under the “Application Server” > “Summary,” you will see a section titled Web Applications. Under this header there will be two apps if you have JRun and ColdFusion set up correctly. They will read “CFMX RDS Application” which we are not going to do anything to, and “Macromedia Coldfusion MX,” which we are going to change. If you click on the name of the application “Macromedia Coldfusion MX,” you will see a simple screen that will show you the current context path for the application, which should be “/cfusion” or something similar. If you change it to “/” then your templates will run from the root domain.

With this process, however there are a couple of caveats. You may have to copy all of the coldfusion JavaScript files to a cfusion subdirectory in your applications folder, if you are using ColdFusion forms validation. Also, the images for the administrator will nont appear when you work with the administrator. Accessing the administrator is not quite as straightforward as you might expect, also. A minor change is needed, it obviously no longer needs the “/cfusion/CFIDE/Administrator/index.cfm,” instead it now will use “/cfide/Administrator/index.cfm.” Make sure to make the “cfide” lowercase or it will not work.

Once you have this working, if you already have applications loaded into the “JRun4/servers/cfusion” directory, and they happen to have the same folder name as the ones in your Apache web root folder, then when you call your templates, the server will not know which ones to pick which will have the effect of causing long nights of hair pulling to figure out why your file changes have no effect on the operation of the server. The resolution is simple, do not use the servers directory of JRun to execute your web applications, instead use the Apache web root. You will have to delete any common files between the appliation in your folder within the JRun servers folder, and the Apache web root. Basically just delete your web application from the JRun application folder, and have it only located in Apache's web root, if you haven't already gotten that.

My issue was that both files had the same index.cfm file, and what was happening was that the virtual root was resolving properly, but a cflocation tag that I had in the index.cfm contained within my JRun servers directory was being chosen over the same file in my Apache web root. Once I deleted the version of the application in the JRun folder, the issue disappeared, the server was behaving correctly.

The moral of the story, don't leave superfluous files around your server, they will always come back to haunt you in the end.


What is this Y!Q stuff?

Posted: December 31st, 1969 | Author: | Filed under: JavaScript, Programming, Uncategorized | Tags: , , | No Comments »

What is this Y!Q stuff?

Picture of Irv Owens Web DeveloperYou may have noticed all of the Y!Q links everywhere on my site. It is a new beta product from Yahoo! that allows people to perform web searches constrained by selected content from the page they are searching from. The content that goes to Yahoo! is selected by the publisher and targeted to return even more relevant results than would be possible going directly to the search engine.

When a user visits a search engine, the system has no background about the person to constrain their results so it makes it difficult to perform a search, for example if I knew someone were from Washington State, and they typed in the word apple, then I could assume they might be looking for apple wholesalers, or apple growers, or apple trees. If someone from California searched for the word apple, I might return the company. This is possible if you know something about the person who is searching, which is why personalized search has been receiving more focus of late.

I prefer the context based approach, because then I don't have to provide any personal information for the search engine to give me what I want. It would know just by the content of the web page that I am searching from.

I'll be honing the coldfusion parsing scripts to give the best possible content to Yahoo! I'll be removing words that are less than four characters in length from the article, to get rid of parts of words and words that carry little meaning like 'the.' I hope to have the best, most relevant results, because Yahoo! is offering $5,000 in their contest. Of course there had to be some motive for me to use this beta program!

I suppose that in its final iteration, Yahoo! will create some type of advertising revenue sharing model similar to Google's adwords. They seem to be hoping that it will generate more clicks because of its usefulness to the user. It is still kind of buggy, for example in all browsers other than Safari 2.0 a semi-transparent overlay pops up when the Y!Q link is pressed, on Safari, it takes you to Yahoo's relevant results page. Hopefully they will fix this soon, I'm pretty sure it has something to do with the changes Apple made to Safari's javascript processing engine. Also, since I am trying to automate this, sometimes a character gets into the string, and causes the Y!Q to return something not valid. I hope this will help with your searching.


JoostBook – Joost to Facebook Interface Widget

Posted: December 31st, 1969 | Author: | Filed under: java, JavaScript, Programming, Uncategorized | Tags: , , , | No Comments »

JoostBook – Joost to Facebook Interface Widget

Picture of IrvinSince I'm in love with Joost, I have been thinking about good applications that I could write for the platform. Before I get into talking about the widget / plugin, let me just say that the experience I have had with communicating with the Joost engineers, through their joost-dev google group, as well as them allowing early access to their SDK, has been outstanding. I have rarely come across a more open and generous group. Typically, the SDK guardians are very selfish about discussing future features, and are usually quite arrogant about the possibility of a developer finding an undiscovered bug. None of this has been the case with the Joost SDK staff.

If you don't want to read the details about how I built it, and you just want to use it, you can get it here: JoostBook: Joost / Facebook Interface. You will need Joost, and a facebook account to get started.

Now, about the widget. Firstly, the installation is a little wierd because of the level of control facebook insists on. In order to use the SDK, you have to authenticate, if an unauthenticated request is made, the response is with the facebook login page. This makes for some unique error catching conditions.

Secondly, we web developers often take for granted that the DOM will have a listener attached to it, and will automatically refresh if anything in the DOM changes. Well, I know that the Joost engineers are working on it, but it doesn't refresh, and therefore, while you can create new XHTML elements, as well as modify the ones that are there with JavaScript. You are best off currently just hardcoding all of your objects up-front, and changing their contents. Also, injecting XHTML using innerHTML doesn't really work so well currently either. I'd suspect that much of this is because there is a bridge between the 2D world of XULRunner / Mozilla, and the 3D world of the Joost interface. I'm sure there is a lot of complexity between the two.

So basically, once you have downloaded Joost, and installed the plugin, the first thing I had to do was check for if you are logged in, if you aren't logged in, it has to show you the facebook login page in an iframe so that the XULRunner browser can be cookied. After that, the widget should work like one would expect. You may have to log in alot, and if you aren't logged in, obviously the application can't update the JoostBook facebook application.

Writing the Joost plugin was the easy part, getting the facebook stuff to work was the hard part. Most of it was because the error handling is terrible. Since facebook doesn't allow you to see the 500 errors that your server is throwing, and it doesn't log it, you have to find other ways to check to see if your server is behaving properly. I spent a lot of time in my logs checking for errors.

The install process is a little wierd too, for example, in Firefox 2.0.0.8 on Windows XP, when I clicked on the Joda file linked in the page, it tried to open it as if it were some kind of markup file, obviously the joda looked like garbage, I had to right click and save. Perhaps if I had used a joost:// link it would have worked OK, but I think more research is in order. I didn't really try it in IE because most of the readers of this blog use Firefox, but it should work the same way.

Then having to install the application in facebook can be a little difficult as well. Well, the installation isn't difficult, its the concept that you have to install two applications that work together that is hard. At least there is no particular order in which you need to install them, worst case whenever you run the JoostBook plugin in Joost, it'll show you the facebook login page all the time.

At any rate, it was a fun experience, and I still think the guys at Joost are on to something. I'm slightly less psyched about the facebook platform, but I'm still excited about it.