Requirements for new Shogi game format

There seems to be great need for a single computer standard for exchanging Shogi games data.

There has been ongoing debate on discussion board (shogi-l) for years now.

Although the discussion tends to be difficult (like the never ending dillema "which came first, the chicken or the egg") but the point is quite simple. After scanning the post you notice that there are just a few simple requirements to be fulfilled.

I try to collect them here. The standard should let for the following features:

1. Storing multiple games in one file.
2. Providing game information (type of the game, players, date, event, outcome, ...) in the way that any shogi software could correctly interpret it.
3. Allowing to store variants.
4. Allowing to store comments in different languages.
5. Allowing to embed multimedia contents (board markups, pictures, sounds, movies) in the comments.
   

Except for the last (embedding multimedia) I consider them to be the core features. They have to be included into the specification. In my opinion no matter if the standard is totally new or if it's based on some existing format (Simple Game Format, Portable Shogi Notation, KIF) it needs to address the problems mentioned above.

By the way, they seem to be easy to implement:

Ad 1) If the single game has concise and well defined description, the games are easily distinguishable/separable. Even if it is not the case, some game separator could be introduced.

Ad 2) The information could be provided in key/value pairs (let's not bother about their form now), for example sente:Sato Yasumitsu or [sente "Sato Yasumitsu"].

We would have to agree on the standard set of tags (sente, gote, event, etc.). The less, the better. The standard, to be flexible and extensible, would state that other tags are allowed but are not given any special meaning. This way if one program needs to add some information that matters only to it, it uses his own tag without any side effects on other programs.

Ad 3) There are many possibilities here. SGF and PSN uses tree-like text data structure for example. I could think of few more solutions but the actual implementation doesn't really matter. Ideally, the structure should be easy for the machines to decode but still easy enough for humans to read it.

Ad 4) For comments we should opt for standard Unicode encoding like UTF-8.

Ad 5) I have some reservations about number 5. It will greatly improve viewing the games with software but it also will make the format less readable by humans.

The most important thing is to make the multimedia comments/markups optional. I imagine not every Shogi game viewer will want the feature (furthermore I believe some target devices, like simple cell-phones, won't be able to use it).

I would split the problem of multimedia comments into two categories:

1. markups that don't require embedding additional data (markups for actions that are understand by viewer: drawing arrows, coloring board fields, showing hyper link, etc.)
2. markups that require embedding additional data (pictures, sounds, movies, etc.)

The first category is easy. The more I think about the second category the less I am convinced it belongs to the standard ;-)

And what do you think?

Learning from our Go colleagues

Trying to figure out the way to increase popularity of Shogi I also look for already proved solutions.

I took a close look at Go world. The game, much like Shogi, is of Asian origin. And it is quite popular all around the world.

It quickly turned out that there are quite a few thinks to be envy about. Few things and activities in the Go community.

I'll try to describe them shortly here as an entry point to the discussion if the ideas would work in Shogi world.

SGF

SGF is the abbreviation of 'Smart Game Format'.

It is the least important of the things I bring up here, but still. Go guys have worked out a standard for storage and presentation of their games. (Of course, as our world is not perfect, there are few flavours of the standard.) The format allows them to discuss comment and discuss games on the net and off-line.
SGF provides many features such that strongly suppor this:

• board markup,
  
• comments,
  
• game information,
  
• setup positions,
• variations etc.
  

You can, for example, draw arrows indicating lines of attack, markup some pieces that play a special role in a given position, etc.

Like I said, the standard itself may not be a big thing, but it let's them to communicate more easily and it also "gave birth" to many game viewer and editor programs.

The Go Teaching Ladder

Now, here is a thing I personally admire a lot. As the site states itself, after several high dan players have recommended getting your own games reviewed by stronger players as a good way to make progress, some guys came up with idea to do it over the net.
Volunteers in the ladder comment games made by weaker players. Anyone may submit a game for commenting by a stronger player.
The service is totally free and community based. This is possible, because many advanced Go players are eager to teach weaker players.
This way weak players benefit from their advice and advanced player benefits from teaching as well.

Isn't it something?

Sensei's Library

Sensei's Library is meant to be a place where Go players can meet to find information, contribute information and discuss any items related to the game. This is a Wiki which everybody is welcome to edit. There are problems with it but I think the idea is great.

There is wealth of information there, whether you are looking introduction to the game, some player's bio, problems to solve -- it's here.

goproblems.com

goproblem.com is a base with problems to solve on-line (currently 5467 problems).
The problems are divided by genre and difficulty. The system measures two types of difficulty and it's "coolness". The difficulty is expressed as: x/y where x is the percentage of people who've gotten this problem wrong and y is the average number of seconds it took people to solve the problem correctly. Coolness is a way for people to judge the problem subjectively.

While you solve the problems your rating is recalculated. You have also access to statistics of your progress.

Another cool thing is that the system let's its users to dynamically shape the database itself. You can do it by adding new problems.
Problems submitted by users wait in the sandbox. Other people play with the problems there and you, as an author, can read people's comments to see if there are any issues with your problem. Also, an examination of the attempt paths will show you where some areas may need to be improved (if a common attempt path has not been accounted for in the problem).

Would it work for Shogi?

Here are my thoughts.

Having standard for game storage and annotation would be useful but is not essential.

Teaching Ladder is a great idea. Unfortunately I think it won't work for us. The community is too small.

Site like Sensei's Library would be great. There is often a situation with "normal" sites that the author looses interest in maintaining it (or doesn't have time, etc. etc.). With Wiki this would be not a problem as anybody (or at least "more people") can make the site live.

As for goproblems.com, people claim to gain much play strength by solving the problems systematically. I think the same is true for Shogi.
I'd love to have this kind of server for Shogi. In fact, I'd want it so badly that I think I could write one myself ;-)

What do You think?

Links

SGF definition -- http://www.red-bean.com/sgf/
Go Teaching Ladder -- http://gtl.xmp.net/
Sensei's Library -- http://senseis.xmp.net/
Go Problems -- http://www.goproblems.com/

What we can do to make Shogi more popular?

Shogi is a great game. No doubts about it. But Shogi is mainly played in Japan. There are some, maybe thousands, enthusiasts outside Japan.
Where this disproportion comes from?
The Japanese group is naturally highly privileged. Shogi is native to Japaneses. There is wealth of information for them: books, theoretical materials, web sites, online game servers. In presence of such an obvious and natural division I will try consider the subject separately for both groups. But before that, let us consider what makes people attracted to the game.

What makes us attracted to the game

I think there are several factors that make an ordinary man a board game fan. I am new to Shogi but I can judge from my experience with chess.

As I contemplate the matter I become to think that on of the most important things is a hero. We like to have someone we admire, whose steps we follow, who is an example for us.
In chess we had our Titans. Capablanca, Tal (the magician of Riga, genius tactician and attacker, my favourite), Alekhine (timeouted to have his 6 teeth extracted during world championship match to come back and win it!), Fisher (great individuality, the man who broke Soviet chess domination lasting for decades), Kasparov (great charisma, some claim that he is the best chessplayer of all time).

I am sure there are plenty of Shogi Heroes out there. But I cannot read a single line about them. Except news posted on discussion board by Mr. Manabu Terao or Mr. Reijer Grimbergen (sorry if I failed to mention other people that try to help us in similar way) there is no information in English.

So, heroes was one thing. The second thing is the community. We need to share our joy from the game. We want to hear how others are doing, what's the latest news in 'our world'. We want to share tips, how to improve. We want to meet to play a game or two (over the board or via Internet). I think there is no need to dwell on this one -- it's quite obvious.

The third think is the struggle for improvement. We love to prove to others that we are good at our game. And we have a chance to do it in very civilized way: by winning a game. There is healthy competition between us and that's good. But there is even more. We struggle with ourselves to get better. We learn from our mistakes, from books, from the articles on the net.

But wait. How can we learn from our mistakes when there is virtually no Shogi software to store and browse our non-Japanese games? How can we read articles when we don't know the language?

There are probably many more reasons that make us interested in a game. Some more important then the others. I just mentioned the reasons I consider most important. I also tried to point out what blocks the factors to make the game of Shogi more popular outside Japan.

I think there something we can do about it.

So, what Japaneses could do?

First of all, make the information flow. Let us know what is going on in Shogi world? How the fight for dominance is going between masters? How our heroes are doing?
The Japan Shogi Association (JSA, http://www.shogi.or.jp/) page is Japanese only. If I am not wrong (and I cannot confirm it, because there is no English information about it) it is the main Shogi federation. I think it is reasonable to require international version of the site from them.
I don't know if there are any, but chess world has few news sites. The most notable, in my opinion, is chessbase.com. Similar site for Shogi would be great.

Second thing is: share your knowledge. There are people doing this already -- you can see their effort on http://shogi-shack.net/default.aspx, http://shogi.typepad.jp/eweblog/ or http://www.teu.ac.jp/gamelab/SHOGI/shogipage.html.

The third thing: translate some software.

The forth: play Shogi with us. Kick our butts so we can learn new ways of doing it.

So, what Gaijins could do?

Well, we could learn Japanese. I never tried it myself but I think it's is quite reasonable. Shogi is a part of Japan culture and it's an opportunity to get it now better.

Before we learn Japanese, though, we could do one simple thing. We can ask our Japanese friends to do all the things we think could help spread Shogi over the world (see above). We could write to JSA and ask them to translate their site.

And, of course, we can try to help ourselves. Once there was this great effort, the Shogi portal called shogi.net. It seems to be dead right now (last update December 2004). Why don't we revitalise it or try something similar?

From what I see on shogi-l, there are some folks that are eager to write some new Shogi software. If there is any way to help, we should try.

OK, that's all for this short divagation.

Any comments are welcome...