Saturday, January 15, 2011

http://shogi-lab.net

Yesterday I came across this site by Adam Szlachta: shogi-lab.net.

It looks like a work in progress but it looks very promising. The site, as I understand it, intends to present in  visually pleasing way, shogi games, openings, tsume, etc.

Like I said, it looks like a beginning now, but have a look at http://www.shogi-lab.net/oneye/.
You can browse through two games from Yoshiharu Habu "Habu's Words".

I hope the site will achieve its full potential...

Best wishes, Adam!
Posted by at 1 comment:
Labels: ,

Saturday, April 18, 2009

Google App Engine and File Upload

Google App Engine (GEA) finally got Java support. That's good. This is leaving me with no more excuses for delaying my Shogi Tools work.

So I got a grip and wrote some more code. Right now I am focusing on turn-based Shogi server.

I wanted there to provide the functionality to upload a user's picture.

But how to achieve this in GAE? GAE doesn't allow you to write files on the discs. How to do this fast and easy? I remember using in some of my earlier projects a ready to use Java library that made file uploading very easy for me.

Apache Commons FileUpload

Thank God for people from Apache.org. I went to their site http://commons.apache.org/fileupload/. The folks prepared a library to add robust, high-performance, file upload capability to our servlets and web applications. I decided to see how would it play with GAE.

Below there is a description of the process of creating a Java web application, under Eclipse environment, which uses commons FileUpload.


Example Google App Project
I assume you have downloaded and installed Google Plugin for Eclipse. In my case I used Eclipse 3.4 (and the dedicated plugin).

We'll start off with creating new Google App Engine project.

Open up your Eclipse. Then create new GAE project.
You can do this by clicking g+ icon on Eclipse toolbar () or choosing File->New->Other->Google->Web Application Project. "New Web Application Project" dialog shows up.


For the purpose of this "tutorial", provide the project name ("UploadTest"), package name ("org.fbc.uploadtest") and uncheck the GWT support. After clicking "Finish" GEA Eclipse
plugin creates a new project for us with a structure similar to the following screenshot:

HTML page for file upload
Next, we create a HTML file which enables the user to pick up the files to upload to the server.
Do this by replacing contents of (generated by GEA plugin) index.html. Open it for editing and paste the following content:


<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>GAE file upload test</title>
</head>
<body>
<form name="filesForm" action="uploadtest" method="post" enctype="multipart/form-data">
File 1:<input type="file" name="file1"><br>
File 2:<input type="file" name="file2"><br>
File 3:<input type="file" name="file3"><br>
<input type="submit" name="Submit" value="Upload Files">
</form>
</body>
</html>

There are few key points here:
  • form's attribute "method" is set to "post"
  • form's attribute "enctype" is set to "multipart/form-data"
  • form's "action" points to the servlet that will handle the upload (yet to be created)
  • input filed type is "file"


Configure Apache commons in Eclipse
First, if we haven't done it yet, we should download FileUpload and IO libraries from Apache's page. I decided to use the latest (1.2.1) version of FileUpload (http://archive.apache.org/dist/commons/fileupload/). The version is dependent (see http://commons.apache.org/fileupload/dependencies.html) on version 1.3.2 of IO library, so I downloaded exactly this version (http://archive.apache.org/dist/commons/io/binaries/commons-io-1.3.2-bin.tar.gz).

After unpacking the contents look for commons-fileupload-1.2.1.jar and commons-io-1.3.2.jar. Copy the files to war/WEB-INF/lib directory (see the project structure image above). This directory contains all the java libraries that will be deployed to appspot server for your app to use them.

Now, for your project to "see" them, you have to add the to the project's build path. I do this by right-clicking on the projects node in the project explorer, choosing Build Path->Configure Build Path (see the image below).


Then I go to "Library tab" (1), click Add JARs (2), indicate desired files in "war" directory (3) and add them to the build path (4). Look below.

The libraries are now in the project's classpath. We can start creating the servlet.


Upload servlet
We are ready to send the file. Now we must prepare ourselves to receive it on the server side. For that purpos we change the contents of "UploadTestServlet.java". GAE plugin generated this class for us in src folder and org.fbc.uploadtest package (see the picture with project structure).

Our upload form (index.html) says it sends the data to the server with "post" method. So, in our servlet we must provide doPost method. Here is the code:

public void doPost(HttpServletRequest req, HttpServletResponse res)
throws ServletException, IOException {
try {
ServletFileUpload upload = new ServletFileUpload();
upload.setSizeMax(50000);
res.setContentType("text/plain");
PrintWriter out = res.getWriter();

try {
FileItemIterator iterator = upload.getItemIterator(req);
while (iterator.hasNext()) {
FileItemStream item = iterator.next();
InputStream in = item.openStream();

if (item.isFormField()) {
out.println("Got a form field: " + item.getFieldName());
} else {
String fieldName = item.getFieldName();
String fileName = item.getName();
String contentType = item.getContentType();

out.println("--------------");
out.println("fileName = " + fileName);
out.println("field name = " + fieldName);
out.println("contentType = " + contentType);

String fileContents = null;
try {
 fileContents = IOUtils.toString(in);
 out.println("lenght: " + fileContents.length());
 out.println(fileContents);
} finally {
 IOUtils.closeQuietly(in);
}

}
}
} catch (SizeLimitExceededException e) {
out.println("You exceeded the maximu size ("
+ e.getPermittedSize() + ") of the file ("
+ e.getActualSize() + ")");
}
} catch (Exception ex) {

throw new ServletException(ex);
}
}

The code should be quite self explanatory, so I won't dwell on this (although I am open to questions :-) ).

I'll just point out few things. I marked two places in the source bold.
The first line:
upload.setSizeMax(50000);

sets the file size limit. When the user tries to upload a file larger then (in our case) 50000 bytes, an exception occurs. When setting the maximal size remember about the GAE limits and quotas.

(If you paste doPost to the class, Eclipse should automatically add imports the code uses. If not so, do add the following lines manually:
import org.apache.commons.fileupload.FileItemIterator;
import org.apache.commons.fileupload.FileItemStream;
import org.apache.commons.fileupload.FileUploadBase.SizeLimitExceededException;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.io.IOUtils;
)

The second line
fileContents = IOUtils.toString(in);
does all the magic needed to fetch the input stream to String. With a single line of code. Thanks, IOUtils! ;-)

Now you have a String containing uploaded file's data waiting for you to do whatever you want to do with it.

In my example I just output the string to the browser (so when you test it it's best to use text files).


Testing the code
All is set and ready. Just run the code. I do this by right-clicking on the project root node in the project browser and choosing Run As->Web Application.


On the Eclipse console, if everything went fine, you should see the following message:

The server is running at http://localhost:8080/

So, open up your favourite browser and go to the address. Choose some files (remember the size limit) and submit the form.

In my case (I chose log4j.properties from src folder), I got the following message in my browser:

--------------
fileName = log4j.properties
field name = file2
contentType = application/octet-stream
lenght: 1063
# A default log4j configuration for log4j users.
#
# To use this configuration, deploy it into your application's WEB-INF/classes
# directory.  You are also encouraged to edit it as you like.

# Configure the console as our one appender
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d{HH:mm:ss,SSS} %-5p [%c] - %m%n

# tighten logging on the DataNucleus Categories
log4j.category.DataNucleus.JDO=WARN, A1
log4j.category.DataNucleus.Persistence=WARN, A1
log4j.category.DataNucleus.Cache=WARN, A1
log4j.category.DataNucleus.MetaData=WARN, A1
log4j.category.DataNucleus.General=WARN, A1
log4j.category.DataNucleus.Utility=WARN, A1
log4j.category.DataNucleus.Transaction=WARN, A1
log4j.category.DataNucleus.Datastore=WARN, A1
log4j.category.DataNucleus.ClassLoading=WARN, A1
log4j.category.DataNucleus.Plugin=WARN, A1
log4j.category.DataNucleus.ValueGeneration=WARN, A1
log4j.category.DataNucleus.Enhancer=WARN, A1
log4j.category.DataNucleus.SchemaTool=WARN, A1

So, everything works fine. The browser uploaded a file to our application, which now could store the file in the GAE datastore.

Hope this info is helpful to anyone.

See you next time,
fat bold cyclop

P.S. I left working example on http://100.latest.kbguesttbook.appspot.com/.
Posted by at 40 comments:
Labels: , ,

Saturday, July 5, 2008

JavaScript ANTLR target

Tonight I managed to build ANTLR version with JavaScript target.

I must admit it was one of the most bizarre builds I did. The problem was not the target itself. The maintainers did a decent job making it. Although it is in it's early stages (the guys claim it not yet ready for publication) the compilation of the target went smooth. Building ANTLR is another story. The instructions (http://www.antlr.org/wiki/pages/viewpage.action?pageId=732) are less then obvious and only a little bit helpful. And that's a bit strange, because all in all, ANTLR is quite well documented.

[[In situation like this I start to appreciate build tools that are more strict as for defining dependencies in projects, like Maven.]]

Anyway, I did build it, so there is no need to complain.

What is important, that I can theoretically use the Shogi grammars I am working on, to build example browser-side parser for game record. Of course, I could do it earlier, by hand. But it would be much more tedious work. And this also gives me an opportunity to learn new things which is, as usual, great fun :-).

I think my first attempt will be a parser/viewer for format in which Reijer Grimbergen posts games on shogi-l and (most of the time) on his pages.

You can see an example game score in this format here. There are no variations. The file consist of only 3 types of information: tag (game details), move (what move whas made) and comment. The file is well formed -- there are strict rules to determine which line is what. A tag line starts and ends with '[' and ']' respectively. A line with move information starts with integer, followed by a dot, followed by move description, followed by two timestamps. The only exception are the comments. They can appear anytime in the text and couldn't be easily distinguished by the machine.

As I understand it, if I, in my parser, write a rule to recognize comment, which is basically free text, the rule would also consume other part of the file also. In other words every line of the text would satisfy the rule (every line can be treated as a free text), so the parser would be confused which rule to apply.

Actually I am not sure if a grammar file can describe it. (read: I am not sure if I could describe it ;-) ). Well, we'll see about that.

Ok, wish me luck.
Posted by at 1 comment:
Labels: , , ,

Thursday, July 3, 2008

JavaScript SFEN parser

Earlier in my posts I described Shogi FEN notation. I also showed SFEN parser -- a Java Web application that creates diagrams from SFEN data.

This time I spent some time learning JavaScript and programing the browser. As a result I came up with JavaScript Shogi Board Viewer and Shogi FEN parser. They have similar functionality as the former, but it's easier to use it on one's own Web pages.

The viewer
JS Shogi Board Viewer is a JavaScript component that makes it possible to generate Shogi position diagram on your Web page.

The viewer consists of 3 elements:
  1. JavaScript program to generate the diagram
  2. CSS file containing style definitions for the diagram
  3. Images representing the board and the pieces
"Interesting" thing about the viewer is the way it is provided with Shogi data.

Well, I wanted my viewer to be "generic" (independent of the position data source) and possibly small. Therefore I decided that the position is provided with a javascript object. Feels like real pain, doesn't it? But fortunately, JSON notation comes to the rescue. Thanks to it we can build JavaScript objects that are understood both by computers and humans.

Besides, JSON is quite popular data exchange format, so there could be converters (i.e. from Shogi FEN or others) easily plugged in to the viewer.

Shogi position data format

The good thing about JSON is that it's very self-descriptive format. Here is how I designed the format.

The description consists of 4 elements:

  • side on move
  • white pieces in hand
  • black pieces in hand
  • pieces on the board

and could be expressed as JSON like this:

{
"sideOnMove":"black",
"blackHand":[],
"whiteHand":[],
"pieces":[]
}

Side on move is simple: it can be either "black" or "white".

Pieces in hand data is an array of numbers. Each number represent the count of pieces for a given rank. The rank is indicated by position in the array. Pieces info is stored in a given order:

  • Pawn (index 0)
  • Lance (index 1)
  • Knight
  • Silver
  • Gold
  • Bishop
  • Rook (index 6)

So, for a Silver and 2 Pawns in hand the data would look like this:

[2,0,0,1,0,0,0,0,0,0]

I am not particularly happy with the format. I don't like the fact that a human has to remember position of piece ranks in the array. The format could be therefore change in the matter. [[Maybe better would be to have pairs rank:count?]]

Pieces on board are given by field:rank pairs. A field is described by 2 chars: column number and row char in the standard Shogi coordinate system. Leftmost column is named '9', rightmost is '1'. Uppermost row is 'a', lowermost is 'i'.

A piece symbol is "borrowed" from Shogi Ladder notation (http://www.shogi.net/ladder/shogiboard.html). The 'w' and 'b' in front of the piece symbol's abbreviations indicate white and black respectively. When a piece is promoted, it will be given with a preceding "+", as in "+bL" or "+wP".

Summing up, here is an example Tsume Shogi position and it's JSON object representation:

tsumeJson = {
"sideOnMove":"black",
"blackHand":[0,0,0,0,1,0,0,0,0,0],
"whiteHand":[0,0,0,0,0,0,0,0,0,0],
"pieces":[{"2a":"wK"},{"1a":"wL"},{"3c":"+wR"},{"1c":"+bB"},{"3d":"+bB"}]
};

The parser

JS SFEN parser is a JavaScript class that is able to translate generate Shogi position data in SFEN to JavaScript objects. The parser and it's best friend, JS Shogi Board Viewer, make it easy to put a Shogi diagram for given SFEN on your Web page.

I won't explain SFEN here. I did my best on this ShogiTools wiki page so If you are interested in the details, please check it out.

The viewer consists of 2 elements:

  1. JavaScript SFEN parser class.
  2. JavaScript Shogi model classes (SFEN is translated to these objects).
Usage of the parser is simple. You put the following line in your code, to get JavaScript object representing given position:
tsumeJson = SfenParser.parse("7r1/6B1p/6Bsk/9/7P1/9/9/9/9 B 2S");
Then you send it to the viewer and voila!

Please, check out the example to see it in action. You can see a Shogi board here and you are able to dynamically set the SFEN to be shown.

The code is put inside the HTML file, so you can view the source to see what's going on there.

I hope that someone finds it useful...

As usual, any comment are welcomed.
Posted by at 1 comment:
Labels: , , ,

Friday, June 27, 2008

New ANTLR targets

Till now I used javacc (http://javacc.dev.java.net/), the Java Compiler Compiler, for creating my parsers.

Like I said earlier (http://shogi-software.blogspot.com/2008/01/tsume-shogi-parser-or-how-to-store-28k.html) I was hoping to try ANTLR out.

Today I read two interesting pieces of information:
  1. There is an IDE to develop and debug grammars, ANTLRWorks.
  2. New (3.1) ANTLR would support JavaScript and ActionScript as targets (platforms in which parsers generated by ANTLR can run).
It is certainly big motivation for me to check this tool out. First of all, ANTLRWorks will make designing and testing my Shogi grammars much easier and faster.
Secondly, new targets will allow creation of shogi games browsing/viewing software on these two very popular platforms: JavaScript and Flash/Flex.

I am going to give it a try very soon.
Posted by at No comments:
Labels: , , ,

Tuesday, June 24, 2008

Visiting the Dark Side

I've been inactive (as a blogger) for quite a time now.

Due to some technical problems MyJavaServer.com didn't accept any new files. That made me search for alternative hosting site for my experiments.
I couldn't find anything similar to mjs but I googled http://www.vndv.com/ out. They give you free hosting for your PHP pages, MySQL databases, subdomains, ftp access, no advertisements, very decent user interface, tutorials. But no Java... Well, nobody's perfect :-)
I spent time on toying a little bit with PHP and client side technologies.

I'll show you my firs PHP page (http://fatboldcyclop.vndv.com/index.php):

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <title>PHP Test</title>
</head>
<body>
  <?php phpinfo(); ?>
</body>
</html>

I am very proud of it :-). But seriously, I did some reading on PHP and I found it interesting. I decided to check it out later and take a closer look at client side web development with JavaScript and CSS first.
My first attempt was to write JavaScript Shogi Board Viewer. I'll describe the program on this blog later. Now I'll share with you my impressions on JavaScript (JS)-- it was basically my first real contact with the language.

First of all, I greatly underestimated the language. Now I think that it is a flexible, dynamic and powerful language. You can do many thing using small clever JS "tricks", but...

But developing in JS is for me a road trip through hell. I managed to get used to auto complete feature of Java IDEs. In Java, which is very strict, the tool can "easily" predict what the programmer can type (object properties and methods, etc.). With JavaScript it's not so easy -- it is dynamic, method and properties can be added on the fly, etc. But IDE is only small part of the problem.
The biggest pain was for me browser compatibility. I had a piece of code ruining smoothly on few browsers (i.e. FireFox 2 and 3, Opera, Safari) but IE somehow didn't like a line or two. Figuring out which line was the real challenge.

Debugging JS is so difficult. Thank God for FireBug. It saved me a lot of time and nerves, but it comes only with FireFox. Fortunately, Opera (9.5) and IE 8.0beta also have a "cheaper" version of the tool.

Another annoying thing about client side web development is CSS compliance. I tried to do simple things like having 2 text boxes next to each other. O what fun it is to make it work exactly the same in all the browsers. This one gives you an extra margin between two boxes, other resizes one box, etc. etc.
By the way, I fought with IE7.0 for correct placement of pieces on the board in my SFEN converter. I won the battle but I lost the war. After I finally forced IE7 to show it right, IE8 was released. And of course, the newer version of the browser has it's own way of positioning page elements.

Anyway, I'm glad I gave it a try. I learnt few things:
  • I lack the imagination to think how people did it before the FireFox came to life.
  • browsershots.org is a great tool to test your web design in different browsers.
  • Trying/learning different languages opens your eyes to new possibilities.
  • JavaScript is an interesting and powerful language but using it is not such fun at all.
  • If you want to show HTML code on your blog, you could use this tool to make it possible.
All in all it was a very educational experiment for me -- short walk on the dark side of the web development.
Posted by at 1 comment:
Labels: , ,

Monday, June 23, 2008

Unicode for Shogi characters


To be able to translate Shogi data between formats you have to know possible piece symbols. I collected  the information I could get (mainly from wikipedia.org) in the following table. Then I found (on http://www.rikai.com/library/kanjitables/kanji_codes.unicode.shtml) unicode values for eaech Shogi piece.
Shogi pieces



















































































































































































English name


Unicode

for abbr. Kanji


Kanji


Rōmaji


Meaning


Abbreviations


Symbol


Kanji


Rōmaji


King (reigning)


738B


王将


ōshō


royal general


K


王


ō


King (challenging)


7389


玉将


gyokushō


jeweled general


K


玉


gyoku


Rook


98DB


飛車


hisha


flying chariot


R


飛


hi


Promoted rook (Dragon)


9F8D or 7ADC


龍王


ryūō


dragon king


+R


龍 or
竜


ryū


Bishop


89D2


角行


kakugyō


angle mover


B


角


kaku


Promoted bishop (Horse)


99AC


龍馬


ryūma or ryūme


dragon horse


+B


馬


uma


Gold general (Gold)


91D1


金将


kinshō


gold general


G


金


kin


Silver general (Silver)


9280


銀将


ginshō


silver general


S


銀


gin


Promoted silver


5168


成銀


narigin


promoted silver


+S


全


—


Knight


6842


桂馬


keima


horse


N


桂


kei


Promoted knight


572D or 4ECA


成桂


narikei


promoted laurel


+N


(圭 or
今)


—


Lance


9999


香車


kyōsha


incense chariot


L


香


kyō


Promoted lance


674F or 4EDD


成香


narikyō


promoted incense


+L


(杏 or
仝)


—


Pawn


6B69


歩兵


fuhyō


foot soldier


p


歩


fu


Promoted pawn (tokin)


3068 or 4E2A


と金


tokin


reaches gold


+p


と (or
个)


to


Coordinates
In KIF files the characters are Shift-JIS encoded. Characters used to describe shogi board coordinates and their unicode values are listed below.





































































































description


char


unicode


1st column


１


FF11


2nd column


２


FF12


3rd column


３


FF13


4th column


４


FF14


5th column


５


FF15


6th column


６


FF16


7th column


７


FF17


8th column


８


FF18


9th column


９


FF19


1st row


一


4E00


2nd row


二


4E8C


3rd row


三


4E09


4th row


四


56DB


5th row


五


4E94


6th row


六


516D


7th row


七


4E03


8th row


八


516B


9th row


九


4E5D




Move modificators
The symbols used in Kifu to identify the piece to move (when there is
a situation in which more then one piece of the same kind can reach
given destination):




































































symbol
unicode
meaning

直
76F4
move straight

引
5F15
pull (back)

上
4E0A
go forward (literally means “up”)

寄
5BC4
go to the side

右
53F3
right

左
5dE6
left

右引




right-back

右寄




right-go to the side

右上




right up

左引




left-back

左寄




left-go to the side

左上




left up


I hope someone finds this information useful.
Posted by at 4 comments:
Labels: , ,
Subscribe to: Posts (Atom)