Sunday, January 27, 2008

Installing Oracle XE on openSUSE Linux

Despite what I have read on the net it was quite easy. I decided to put the information on the net in case I would have to repeat the procedure. And maybe someone else would need the information...

First I'll tell you what machine I use, then I'll show you what I did.

The machine

I use HP Compaq 6710b laptop, with 2G of RAM. Inside the machine is the Intel® Core™2 Duo T7300 2GHz processor.
There is Windows Vista preinstalled in it, but I decided to use OpenSuse (10.3) Linux as my development environment.

Installation procedure

You could summarize the whole process in 3 steps:
1. Obtaining the installation file.
2. Installing Oracle XE and required libraries.
3. Configuring Oracle instance.

In my case the procedure went as follows:
1. I downloaded Oracle XE for Linux x86 ( I chose oracle-xe-univ-

2. Logged as the root I run the installer. Everything went smooth except for the database didn't run. I checked the installation guide (I know, I should have done it earlier :-) ) and it turned out that this Oracle distribution requires libgc and libaio libraries.

I ran software management program (/sbin/yast2 --install) that lets you see and modify software components installed on your openSUSE.
In my case the libraries where absent so I looked for them on openSUSE Build Service pages (, downloaded and installed them.

3. The final step, according to the manual was running the provided configuration script. So I did, as the root, ran

/etc/init.d/oracle-xe configure

I chose the default options when prompted and set up the password for sys and system users:
Do you want Oracle Database 10g Express Edition to be started on boot (y/n) [y]:y
Starting Oracle Net Listener...Done
Configuring Database...Done
Starting Oracle Database 10g Express Edition Instance...Done
Installation Completed Successfully.
To access the Database Home Page go to ""
I tried to run the database's home page by running FireFox and entering address.

Unfortunately I couldn't log in (I probably did something wrong but I am not sure what was it -- maybe just misspelled the password during log-in),
so I opened "Run SQL command line" and typed:
conn sys as sysdba ;
alter user system identified by ;
alter user sys identified by ;
After this maneuver I retried entering user/password and finally successfully logged into the database's home page.

Like I said, I am not sure if the last step (resetting the password) was necessary. When I have a chance to run the installation again I'll correct the information.

Have fun with Oracle on openSUSE!

Friday, January 25, 2008

New Shogi Tools banner

A friend of mine was so kind to make a banner for my Shogi Tools project.

I put the banner on the project's page. I also used it on

Thanks Yummy!

Wednesday, January 23, 2008

Learning from our Go colleagues

Trying to figure out the way to increase popularity of Shogi I also look for already proved solutions.

I took a close look at Go world. The game, much like Shogi, is of Asian origin. And it is quite popular all around the world.

It quickly turned out that there are quite a few thinks to be envy about. Few things and activities in the Go community.

I'll try to describe them shortly here as an entry point to the discussion if the ideas would work in Shogi world.


SGF is the abbreviation of 'Smart Game Format'.

It is the least important of the things I bring up here, but still. Go guys have worked out a standard for storage and presentation of their games. (Of course, as our world is not perfect, there are few flavours of the standard.) The format allows them to discuss comment and discuss games on the net and off-line.
SGF provides many features such that strongly suppor this:
  • board markup,
  • comments,
  • game information,
  • setup positions,
  • variations etc.
You can, for example, draw arrows indicating lines of attack, markup some pieces that play a special role in a given position, etc.

Like I said, the standard itself may not be a big thing, but it let's them to communicate more easily and it also "gave birth" to many game viewer and editor programs.

The Go Teaching Ladder

Now, here is a thing I personally admire a lot. As the site states itself, after several high dan players have recommended getting your own games reviewed by stronger players as a good way to make progress, some guys came up with idea to do it over the net.
Volunteers in the ladder comment games made by weaker players. Anyone may submit a game for commenting by a stronger player.
The service is totally free and community based. This is possible, because many advanced Go players are eager to teach weaker players.
This way weak players benefit from their advice and advanced player benefits from teaching as well.

Isn't it something?

Sensei's Library

Sensei's Library is meant to be a place where Go players can meet to find information, contribute information and discuss any items related to the game. This is a Wiki which everybody is welcome to edit. There are problems with it but I think the idea is great.

There is wealth of information there, whether you are looking introduction to the game, some player's bio, problems to solve -- it's here. is a base with problems to solve on-line (currently 5467 problems).
The problems are divided by genre and difficulty. The system measures two types of difficulty and it's "coolness". The difficulty is expressed as: x/y where x is the percentage of people who've gotten this problem wrong and y is the average number of seconds it took people to solve the problem correctly. Coolness is a way for people to judge the problem subjectively.

While you solve the problems your rating is recalculated. You have also access to statistics of your progress.

Another cool thing is that the system let's its users to dynamically shape the database itself. You can do it by adding new problems.
Problems submitted by users wait in the sandbox. Other people play with the problems there and you, as an author, can read people's comments to see if there are any issues with your problem. Also, an examination of the attempt paths will show you where some areas may need to be improved (if a common attempt path has not been accounted for in the problem).

Would it work for Shogi?

Here are my thoughts.

Having standard for game storage and annotation would be useful but is not essential.

Teaching Ladder is a great idea. Unfortunately I think it won't work for us. The community is too small.

Site like Sensei's Library would be great. There is often a situation with "normal" sites that the author looses interest in maintaining it (or doesn't have time, etc. etc.). With Wiki this would be not a problem as anybody (or at least "more people") can make the site live.

As for, people claim to gain much play strength by solving the problems systematically. I think the same is true for Shogi.
I'd love to have this kind of server for Shogi. In fact, I'd want it so badly that I think I could write one myself ;-)

What do You think?


SGF definition --
Go Teaching Ladder --
Sensei's Library --
Go Problems --

Friday, January 18, 2008

Shogi FEN -- similar works

Different FEN

After publication of the post about my proposal for Shogi positional data storage standard, there was some comments on shogi-l.
Bernhard C. März pointed out that there is similar way of storing Shogi positions described earlier. He gave a link to the archived shogi-l message:
The description differs from mine in 'pieces in hand' part.

There was some discussion about the approach.

>> > *fat bold cyclop* wrote:
>> > > I think the only flaw of it is, I think, it insufficient for
>> > > storing positions with more than 9 pieces in hand.
>> > In my opinion, this is a rather serious problem.
>> Just use A, B, C, ..., H for 10 through 18.
>> Using A through F for 10 to 16 is pretty standard (hexadecimal).
>> This particular extension seems rather obvious.

I have implemened a FEN notation in MacShogi 2 years ago, but I don't
have added the GUI part to use it yet.

I first have chosen the hexadecimal option to store the number of
pawns in hand but finally, I have preferred to use only digits.

The algorithm is pretty simple for pawns: the pieces in hand part in
the notation has 7 digits in all, one digit for each piece type,
except for the pawn where 2 digits can be used. So, if the string
contains 8 digits, the pawn number uses 2 digits, otherwise only one
digit is used for the number of pawn pieces.

It works well. But this is only a matter of personal choice.

Another SFEN

Tord Romstat devised (draft dated 2007-01-24).
Both approaches seem to be identical. Both in contents and name :)

Sunday, January 13, 2008

Proposal: Standard for storing Shogi position information

Foreword: I know that similar work have been undertaken by different people already.
I don't claim I am inventing something that others didn't think of before. If you know some sources I'd be glad to put links to them in this post.

That said, I move on to the subject with is a proposal of standard for storing positional data in Shogi. I called it Shogi FEN (or SFEN) because it is derived from FEN standard designed for chess. I write more about the standard itself on ShogiTools project's wiki. Here I try to advocate the need for a standard like SFEN.

Do we need such a standard

I think it is very important to have a standardized way to write Shogi positions. This way data are easily exchanged between programs. Let me give you some examples:
  1. Imagine you have a database program that stores tsume Shogi problems. Your computer is Mac with English OS, your friend has Japanese Windows and he uses different database program and you want to share the problems with him.
  2. You post some positions from games of masters as a diagram on your web page using some flash application. The application is faulty and you want to switch to other program, let's say Java applet.
  3. You want to have a test suit of 100 positions for benchmarking different Shogi engines.
In all these situations, if there is a standard way to store positional data, there is no additional work required on your side. Otherwise there should be some conversion mechanism provided. Moreover, to be able to convert between proprietary standards you have to know them both.

What's so good about SFEN

Nothing :-). At least nothing until it becomes a standard. But what are the 'pros' of it?

It's more compact than traditional ASCII-diagram notation. The whole position data fits a sigle text line. In my opinion they are also easier to interpret by programs and quite readable by humans at the same time.

The idea of similar standard succeeded in chess world. I think we could learn from their experience and adopt similar solution.

Why wouldn't we want SFEN

I can think about only one good reason: "We already have a good standard for this and don't need another one".

If this would be the situation, I probably wouldn't bother making this post. But, as I see it, there are some solutions 'on the market', but:
  1. They are not really standards -- there is many flavors of single 'standard'.
  2. There is no formal specification for them -- at least none non-Japanese.
If you know of some standards, real or almost real, their specification on the net, please drop me a note. I'll be happy to post links to them and discuss them here.

Live demo

I prepared a site where you can see SFEN in action.

It's a nice moment for me, because the puzzles of my ShogiTools project start to fit together. In my earlier post about Tsume Parser I described how I translated tsume problems from shogi-l into computer-readable form.

Here I used the product of that project and made a simple converter to SFEN data.

Finally, I put all (almost 300) tsume on the page. You can see on-line SFEN to diagram conversion (and solve some tsume at the same time! ;-) ).

Note. When runing applications, don't worry if you see something like Can't contact servlet runner at does this kind of things from time to time. Wait few moments and try again. MyJavaServer is down for good. The example has been moved to Google AppEngine.

Note 2. The page is best viewed with Mozilla browser. I could force Internet Explorer to cooperation. The elements are slightly misplaced. I think I finally won the IE battle.

Other works

After publication of the post there was some comments on shogi-l.
Bernhard C. März pointed out that there is similar way of storing Shogi positions described earlier. He gave a link to the archived shogi-l message:

Thank you for your attention. As always -- any comments and suggestions are welcomed.

Monday, January 7, 2008

What we can do to make Shogi more popular?

Shogi is a great game. No doubts about it. But Shogi is mainly played in Japan. There are some, maybe thousands, enthusiasts outside Japan.
Where this disproportion comes from?
The Japanese group is naturally highly privileged. Shogi is native to Japaneses. There is wealth of information for them: books, theoretical materials, web sites, online game servers. In presence of such an obvious and natural division I will try consider the subject separately for both groups. But before that, let us consider what makes people attracted to the game.

What makes us attracted to the game

I think there are several factors that make an ordinary man a board game fan. I am new to Shogi but I can judge from my experience with chess.

As I contemplate the matter I become to think that on of the most important things is a hero. We like to have someone we admire, whose steps we follow, who is an example for us.
In chess we had our Titans. Capablanca, Tal (the magician of Riga, genius tactician and attacker, my favourite), Alekhine (timeouted to have his 6 teeth extracted during world championship match to come back and win it!), Fisher (great individuality, the man who broke Soviet chess domination lasting for decades), Kasparov (great charisma, some claim that he is the best chessplayer of all time).

I am sure there are plenty of Shogi Heroes out there. But I cannot read a single line about them. Except news posted on discussion board by Mr. Manabu Terao or Mr. Reijer Grimbergen (sorry if I failed to mention other people that try to help us in similar way) there is no information in English.

So, heroes was one thing. The second thing is the community. We need to share our joy from the game. We want to hear how others are doing, what's the latest news in 'our world'. We want to share tips, how to improve. We want to meet to play a game or two (over the board or via Internet). I think there is no need to dwell on this one -- it's quite obvious.

The third think is the struggle for improvement. We love to prove to others that we are good at our game. And we have a chance to do it in very civilized way: by winning a game. There is healthy competition between us and that's good. But there is even more. We struggle with ourselves to get better. We learn from our mistakes, from books, from the articles on the net.

But wait. How can we learn from our mistakes when there is virtually no Shogi software to store and browse our non-Japanese games? How can we read articles when we don't know the language?

There are probably many more reasons that make us interested in a game. Some more important then the others. I just mentioned the reasons I consider most important. I also tried to point out what blocks the factors to make the game of Shogi more popular outside Japan.

I think there something we can do about it.

So, what Japaneses could do?

First of all, make the information flow. Let us know what is going on in Shogi world? How the fight for dominance is going between masters? How our heroes are doing?
The Japan Shogi Association (JSA, page is Japanese only. If I am not wrong (and I cannot confirm it, because there is no English information about it) it is the main Shogi federation. I think it is reasonable to require international version of the site from them.
I don't know if there are any, but chess world has few news sites. The most notable, in my opinion, is Similar site for Shogi would be great.

Second thing is: share your knowledge. There are people doing this already -- you can see their effort on, or

The third thing: translate some software.

The forth: play Shogi with us. Kick our butts so we can learn new ways of doing it.

So, what Gaijins could do?

Well, we could learn Japanese. I never tried it myself but I think it's is quite reasonable. Shogi is a part of Japan culture and it's an opportunity to get it now better.

Before we learn Japanese, though, we could do one simple thing. We can ask our Japanese friends to do all the things we think could help spread Shogi over the world (see above). We could write to JSA and ask them to translate their site.

And, of course, we can try to help ourselves. Once there was this great effort, the Shogi portal called It seems to be dead right now (last update December 2004). Why don't we revitalise it or try something similar?

From what I see on shogi-l, there are some folks that are eager to write some new Shogi software. If there is any way to help, we should try.

OK, that's all for this short divagation.

Any comments are welcome...

Thursday, January 3, 2008

Tsume Shogi Parser (or "How to store 28k of data into 802k file and be happy about it")

What is Tsume Shogi Parser?

I am a huge shogi patzer. I am also a poor Java programmer. Well, all in all, I am a miserable creature. But even though I am so old that I remember the era of black&white TV, I think I can improve. (Not in being miserable, I mean. The other way.)

I wanted to strengthen my shogi skills by solving mating problems. You would be surprised how hard it is to find some good materials on the net. If you have some experience with Go or Western Chess you think that the Internet is the ultimate source for this kind of stuff. You couldn't be more mistaken -- at least when you don't read Japanese.

Fortunately, there are good folks out there. Shogi-l email discussion list is a place where people like these meet. Reijer Grimbergen, i.e., posted series of 300 tsume shogi problems. Roger Hare was so kind to collect some of them into a single file:

That's how I got my learning materials. The only problem left was it's form:
4. Black: R4d, +B4c, +P2a In hand: G
White: K2c, G3c, S4b, L1b, P4a

5. Black: R2b, G3e, S3a, L2e In hand: None
White: K3c, G4c, S1d, P4e

A beginner like me could use some more "visually attractive" format. By the way, it's a human nature I think: people always want more. First I complained that there are no materials at all, and then (after I found them) I wanted them to be full of diagrams and nicely formated.
Anyway, that's how I decided to write a converter for the problems. I called it Tsume Shogi Parser.

The process

Tsume Shogi Parser is intended to be a building block to use in other applications (these applications are meant to provide user interface for user convenience). In the near future I plan to provide a web app that would allow the user to upload a text file and get the resulting pdf. The parser could also be used in shogi game database to produce handouts on the fly.

Ok, let's face it. The parser will be not for much use. After all, there is only one source of data stored in this format. I treat this thing as an exercise before more important challenge: writing a parser for "This week in Shukan Shogi" posted by Reijer Grimbergen on shogi-l. The second benefit from it is trying out the part that generates pdf. That's IS going to be useful in other projects.

Let's go back to the parser. For know I’ll describe the inner mechanics of the Tsume Shogi Parser.


For this piece of software I chose to use:
  • Java -- of course ;-),
  • JavaCC -- for defining the input file grammar and automatically generate a parser,
  • XML -- for storing the data into format that can be converted into virtually anything else,
  • JiBX -- for translating Java objects to XML (marshaling)
  • XLS-FO -- for storing information about expected layout of the handout,
  • XSLT 2.0 -- for defining a way of injecting tsume data to layout template,
  • Saxon -- the only XSLT processor I know that supports XSLT 2.0,
  • Apache FOP -- for generating PDF.
So, the process can be diagrammed as follows:

Shogi-l file->(.jj + javacc + jibix)->xml file->(xslt 2.0, saxon)->(fo, fo)->pdf

and in the future

-> spd, psn
-> database

Reading tsume

I have put all the tsume problems I found on shogi-l (most of them posted by Reijer Grimbergen) to a single file. If you're interested in it, you can download it from Tsume Shogi Project download pages. I also collected in a single file the answers for the tsume problems.

Then I wrote a JavaCC grammar for the file -- I made the computer understand what which field in the file meant. You can see the grammar on the ShogiTools project's wiki.

It was my first contact with parsers and it cost me a lot of sweat. I am still no expert here but I manage to write a simple grammar and generate a parser from it. JavaCC is of great help here. It automates most of the work so you can focus on the merit, although I had lots of problems with it. Partly because I was a beginner in the matter of parsers, partly because of lousy project documentation and community support. I've read that similar program, ANTLR, is much better in these matters. Next time I should try it so I could make an honest comparison. (If you can share your experience in the subject, please, drop me a note.

The thing is, that with the help of JavaCC and JavaCC Eclipse Plugin I generated a program that read tsume problem file and stored it in memory in Java objects. (The class diagrams of the value object used for it should be on ShogiTools pages.)
It turned out that the input data was a bit inconsistent in form. The parser showed few places and I had to do some corrections by hand. But this was not a big problem.

Producing XML

Now I wanted the data stored into an XML file.

Why XML? It is license-free, platform-independent and well-supported. Also because of the promise of easy transformations to presentation formats (without coding).

Anyway that's what the theory says. I wanted to see this in practice.

The process of transforming the memory representation of an object to a data format suitable for storage or transmission is called Marshaling. So I typed "fastet marshaller" into google search engine and got JiBX on pole position. And I used JiBX.

The interesting thing about JiBX is it's byte-code injection mechanism. In short: you define the mapping between classes and the XML. Then the mapping information is injected into your existing classes' bytecode. This means you can structure the Java classes the way you want -- they doesn't need to match the structure of the XML. It also turns out that this is a fast way do the conversion.

This part was quite quick. The program read 11k of tsume definitions and spat out 802 kB XML file. The description of XML structure can be found on ShogiTools pages.

The output

So, what is so good about having such a huge file instead of small one? And why it is so big?

Let's compare the entry for a single position in input text file and output xml file.
The text entry goes like this:
1. Black:+B3d, +B1c In hand: G
White: K2a, +R3c, L1a

Equivalent XML fragment is:

The second fragment is huge. It is well structured, data are organized in logical chunks. You may ask here: So what!?

And I agree. It's is no good to have kosher data structures for the sake of kosherness only. The real issue is what you can do with the data. There are the benefits of data conversion. Let me show you how our XML can be transformed.

My first try was to write XSL transformation to get an ASCII diagrams of the positions. For the given position (XML fragment above) I got this:


White in hand:
9   8   7   6   5   4   3   2   1
|   |   |   |   |   |   |   | wK| wL|a
|   |   |   |   |   |   |   |   |   |b
|   |   |   |   |   |   |+wR|   |+bB|c
|   |   |   |   |   |   |+bB|   |   |d
|   |   |   |   |   |   |   |   |   |e
|   |   |   |   |   |   |   |   |   |f
|   |   |   |   |   |   |   |   |   |g
|   |   |   |   |   |   |   |   |   |h
|   |   |   |   |   |   |   |   |   |i
Black in hand:G

The whole file can be downloaded from here.
Surly, we can do better than this! Well, we're almost there :-) This ASCII diagram thing was just a warm up before writing XSLT to produce PDF file. First I had to do xsl-fo my homework. For the PDF renderer I chose apache FOP which is quite well documented. So, I was able to learn some xsl-fo from apache documentation. I also used w3schools tutorials (
I wanted my tsume shogi handout to have 6 problems per page and some tsume definition as an introduction. It was quite a bit of tedious work but I think it was worth it.
Judge for yourself. Here is the output pdf file.

Future development

I have my handouts for doing some shogi exercises off-line. My goal is reached so I stop my experiments for now. Instead I will list possible improvements below.

The pdf I managed to produce has English description of pieces on the board (i.e. wG for white gold, bK for black king). I could use Kanji characters instead. But first I need to know the unicode for each piece character and own some unicode font.

The whole thing is not a big deal. But during the process of converting my tsume text files into pdf I managed to learn few things. I think I also broke fresh ground for future conversion component that could be used in shogi programs. Imagine, you have your shogi database and you are able to add ability to render you games in the format you need just by supplying suitable transformation (XSLT) file.

See You next time...

Tuesday, January 1, 2008

Shogi Tools

Who is Fat Bold Cyclop?

My real-life alter ego works in the software industry, has a beautiful wife, two great kids and a dog.

We both like the same things but when it comes, after hard day of work, he's time to get rest I take over the control and do some fun stuff. These include playing chess or shogi over the net, watching movies, reading books. He is always puzzled why he is so tired :-)

For more information about me, please visit FBC's homepage.

One of the things of our interest is shogi. The other is developing software.

What are Shogi Tools

The whole idea of this appeared when I was browsing shogi-l discussion list.

I was looking for some tsume shogi problems to solve. Finally, I found in their archives a file Roger Hare collected there 131 problems (mostly submitted by Reijer).

I wanted to have the tsume collection in some printable format so I could make a handout.

So I started to write a tool which I called Tsume Parser. It is supposed to read a file in the descriptive format (the format used by Roger) and convert it into more "visually attractive" one.

I also wished I had a program to help me to enhance my "board vision" in shogi. There are some similar programs for chess (i.e. but none for Japanese chess.

I have other ideas for software that could help organize my shogi.
These led me to the conclusion that I have to take active part in developing some shogi software.

That's how Shogi Tools appeared as a project on There are two main goals of the project:

  • Documenting shogi community standards for storing, representing, transmitting shogi-related data (games, tsume, etc.).
  • Building tools to translate between the formats.
Future plans contain, after building mentioned set of tools, production of turn based on-line game server for shogi players.

On the pages of this blog You will be able to trace the progress of my work on the subject. I'll try to describe the problems I encounter and how (or if) I solved them.

See you!