Compiling Brogue v1.6.4 with MinGW

Brogue is a very user friendly roguelike.  It implements a lot of the things that I have always wanted whenever I play a "tile-based" text game of this kind.  That includes support for mouse over events, and giving visual indications of where on the map the selected item is, and the like..

It comes with source code, which should allow me to make my own changes, but unfortunately with programming.. it's never that simple.
In an ideal world, I could just create a new project in Visual Studio, but the code is not platform independent and requires dirent.h and associated functionality.  太麻烦了... so MinGW it is.  The option of using the Code Blocks IDE and it's version of MingW is present, but that's just more complication of a different kind.

1. Have a MinGW/msys compilation environment set up.
2. Open a MinGW shell.
3. Go to the "Brogue Source" directory that comes with the Windows Brogue binaries.
4. Download and extract: libtcod-1.5.1-mingw32.zip (from here).  It is already compiled.
5. Download and extract: SDL-devel-1.2.14-mingw32.tar.gz (from here).  It is not necessary to compile it as the headers are all that are required.
6. Apply patch below for "tcod-platform.c".
7. Edit "makefile".
8. Line 3: Correct CFLAGS with -DBROGUE_TCOD instead of -DBROGUE-TCOD.
9. Line 23: Add the missing "line continuation" character '\'.
10. Line 32: Correct the g++ linking command modifying "-lSDL.dll" to be "-lSDL".
11. Line 32: Correct the g++ linking command adding "-L$(LIBTCODDIR)/" so that g++ can find "SDL.dll" and "libtcod-mingw.dll".
12. Line 32:  Extend the g++ linking command adding "-static-libgcc -static-libstdc++" so that the compiled executable can be run without a dialog box saying "The program can't start because libgcc_s_dw2-1.dll is missing from you computer. Try reinstalling the program to fix this problem.".

This gives you a replacement "brogue.exe" binary which can be copied up to the outer "Brogue v1.6.4" directory, and does not have any further cumbersome MinGW dependencies.

Patch for "tcod-platform.c":

< #include <SDL/SDL.h>
---
> #include <SDL.h>
293c293,295
<               bufferedKey = TCOD_console_check_for_keypress(TCOD_KEY_PRESSED);
---
>               TCOD_sys_check_for_event(TCOD_EVENT_KEY_PRESS | TCOD_EVENT_MOUSE, &bufferedKey, &mouse);
>       } else {
>               TCOD_sys_check_for_event(TCOD_EVENT_MOUSE, &bufferedKey, &mouse);
390c392
<               key = TCOD_console_check_for_keypress(TCOD_KEY_PRESSED);
---
>               TCOD_sys_check_for_event(TCOD_EVENT_KEY_PRESS | TCOD_EVENT_MOUSE, &key, &mouse);

The patch makes up for what seems like custom changes to the libtcod DLL that comes with Brogue.  Out of the box, libtcod does not update the last known mouse state unless the using application polls mouse state.

Edit: Added all the other extra steps I had forgotten to mention, as I have gone through this process many times now.

Android NDK & Windows symbolic links

There are projects out there that have symbolic links in them.  Because they're stored in GIT, you get these files and directories in a corrupted form.  They just don't work, so you write a script to convert them to Windows symbolic links.

#!/bin/sh

git ls-files -s | awk '/120000/{print $4}' | while read FILEPATH
do
    if [ ! -e $FILEPATH ]; then
        echo "Please check that GIT core.symlink for this project is 'false'" 1>&2
        exit 1
    fi

    if [ ! -L $FILEPATH ] && [ -f $FILEPATH ]; then
        wc -l $FILEPATH | while read LINES DISCARD
        do
            if [ "$LINES" = "0" ]; then
                FILEDIR=`dirname $FILEPATH`
                FILENAME=`basename $FILEPATH`
                LINKPATH=`cat $FILEPATH`
                pushd . > /dev/null
                cd $FILEDIR
                if [ -e $FILENAME ]; then
                    if [ -e $LINKPATH ]; then
                        echo "Repairing symlink: $FILEPATH"
                        if [ -f $FILEPATH ]; then
                            rm $FILENAME
                            cmd /c "mklink ${FILENAME//\//\\} ${LINKPATH//\//\\}"
                        else
                            rm $FILENAME
                            cmd /c "mklink /d ${FILENAME//\//\\} ${LINKPATH//\//\\}"
                        fi
                    else
                        echo "$FILEPATH: symlink path '$LINKPATH' invalid" 1>&2
                    fi
                else
                    echo "$FILEPATH: problem finding this file"
                fi
                popd > /dev/null
            else
                # ERROR: Expected a file with one line for the link path
                echo "$FILEPATH: bad link file: expected 0 lines, got $LINES" > /dev/null # 1>&2
            fi
        done
    else
        echo "$FILEPATH: already a symlink" > /dev/null # 1>&2
    fi    
done

You run it within MinGW Shell in the top level directory of the project, but MinGW Shell is run as Administrator so that it has permissions to create Windows symbolic links.  And it appears to work.  You can see the linked files or directories looking like shortcuts in explorer, and you can open them.  But then you try and compile something with the Android NDK.

Compile thumb  : sdl-1.2 <= SDL_androidinput.c
/d/Code/android-ndk-r8/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o.d.org -fpic -ffunction-sections -funwind-tables -fstack-protector -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__  -Wno-psabi -march=armv5te -mtune=xscale -msoft-float -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../jni/sdl-1.2/include -Ijni/../jni/sdl-1.2 -DANDROID -O3 -DSDL_JAVA_PACKAGE_PATH=net_sourceforge_gemrb -DSDL_CURDIR_PATH=\"net.sourceforge.gemrb\" -DSDL_TRACKBALL_KEYUP_DELAY=1 -DSDL_VIDEO_RENDER_RESIZE_KEEP_ASPECT=1 -DSDL_VIDEO_RENDER_RESIZE=1 -DSDL_ANDROID_KEYCODE_0=LCTRL -DSDL_ANDROID_KEYCODE_1=c -DSDL_ANDROID_KEYCODE_2=NO_REMAP -DSDL_ANDROID_KEYCODE_3=NO_REMAP -DSDL_ANDROID_KEYCODE_4=e -DSDL_ANDROID_SCREENKB_KEYCODE_0=LCTRL -DSDL_ANDROID_SCREENKB_KEYCODE_1=c -DSDL_ANDROID_SCREENKB_KEYCODE_2=NO_REMAP -DSDL_ANDROID_SCREENKB_KEYCODE_3=NO_REMAP -DSDL_ANDROID_SCREENKB_KEYCODE_4=e -Wa,--noexecstack -O0 -g -O2 -DNDEBUG -g -I/d/Code/android-ndk-r8/platforms/android-8/arch-arm/usr/include -c  jni/../jni/sdl-1.2/src/video/android/SDL_androidinput.c -o ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o && ./obj/convert-dependencies.sh ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o.d
jni/../jni/sdl-1.2/src/video/android/SDL_androidinput.c:41:30: error: SDL_androidvideo.h: No such file or directory
...

Observing the file access in Process Monitor, access flags it as a directory in some instances. Even though file access appears to work outside the NDK case.  MinGW shell almost believes it works when you use the file command on it.

$ file project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
project/jni/sdl-1.2/src/video/android/SDL_androidinput.h: ASCII C program text, with CRLF line terminators

If you try and copy the file that it links to over top of it, that seems to think they're the same.

$ cp -f project/jni/sdl-1.3/src/video/android/SDL_androidinput.h project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
cp: `project/jni/sdl-1.3/src/video/android/SDL_androidinput.h' and `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h' are the same file

Something shows as broken if you try and delete it.

$ rm -f project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
rm: cannot remove `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h': Not owner

A bit of googling suggests trying the rmdir command.

$ rmdir project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
$ file project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
project/jni/sdl-1.2/src/video/android/SDL_androidinput.h: ERROR: cannot open `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h' (No such file or directory)

It looks like the best approach is to copy files rather than link to them. Linking to directories might be stable, but just because it seems to work for now, doesn't mean that problems might be present in certain use cases. As file linking had the above NDK compilation use case, for instance.

msysgit & corrupted symbolic links

Using msysgit on Windows?

Let's say the project you are cloning has:

real-directory-1/
real-directory-2/linked-directory-1/

Where linked-directory-1/ is a symbolic link to real-directory-1/.

What you will find in your clone of this project is:

real-directory-1/
real-directory-2/linked-directory-1

Where linked-directory-1 is a file containing the local path of what it's target would be if it were the symbolic link it were supposed to be.


The problem?

Cloning projects in a broken state just seemed.. ridiculously unusable.  I've wasted hours on errors that appear to be in one place, but are actually because of these broken symbolic links.  With a bit of googling, I encountered someone suggesting that someone else set their core.symlink GIT setting.  Out of curiosity, I decided to check the .git/config file in my cloned project.

[core]
        repositoryformatversion = 0
        filemode = false
        bare = false
        logallrefupdates = true
        symlinks = false
        ignorecase = true
        hideDotFiles = dotGitOnly

I don't understand..  The default value is to create broken clones, with missing symlinks and junk files littered throughout them?  Is there something I am missing here?

The solution?

A bit of further searching revealed that you can set this setting globally.  Something like:

git config --global core.symlink true

After executing that, and then modifying the red highlighted project-specific setting..

$ git checkout -f
error: unable to create symlink AndroidAppSettings.cfg (Function not implemented)
error: unable to create symlink project/jni/application/atari800/atari800 (Function not implemented) 

Now the files are missing, but at least I know that the clone is bad.

I think the best approach is to have it work this way by default, then cloned projects can have their local setting changed in their configuration file.  Then the corrupted clone can be processed with a script to turn the corrupt files into the links they should be.

The following almost does the trick:

#!/bin/sh

git ls-files -s | awk '/120000/{print $4}' | while read FILEPATH
do
    if [ ! -e $FILEPATH ]; then
        echo "Please check that GIT core.symlink for this project is 'false'" 1>2
        exit 1
    fi

    if [ ! -L $FILEPATH ] && [ -f $FILEPATH ]; then
        wc -l $FILEPATH | while read LINES DISCARD
        do
            if [ "$LINES" = "0" ]; then
                FILEDIR=`dirname $FILEPATH`
                FILENAME=`basename $FILEPATH`
                LINKPATH=`cat $FILEPATH`
                pushd . > /dev/null
                cd $FILEDIR
                if [ -e $FILENAME ]; then
                    if [ -e $LINKPATH ]; then
                        ln -s -f $LINKPATH $FILENAME
                        echo "Repairing symlink: $FILEPATH"
                    else
                        echo "$FILEPATH: symlink path '$LINKPATH' invalid" 1>2
                    fi
                else
                    echo "$FILEPATH: problem finding this file"
                fi
                popd > /dev/null
            else
                # ERROR: Expected a file with one line for the link path
                echo "$FILEPATH: bad link file: expected 0 lines, got $LINES" > /dev/null # 1>2
            fi
        done
    else
        echo "$FILEPATH: already a symlink" > /dev/null # 1>2
    fi    
done

What it does not do, is create a symbolic link within the same directory, to that same directory the link is intended to reside within.

Conclusion

Do not use Git for Windows AKA msysgit.  Even if you work around the corrupted files with the above script, you cannot make the links yourself because some cases are not supported by the ln command.  You can get a little further by switching over to a MinGW msys installation (the above script works in the "does not do" case), but there are still problems.  It is probable that MinGW ln is copying, rather than linking, given the time the script takes to run.

Cygwin has the same problem, when using it's git command.

Portable Ubuntu has been recommended, but I haven't tried it yet, or looked at it's suitability.

Next: Android NDK & Windows symbolic links.

Mnemosyne 2 finally released

Just noticed there was a new release of Mnemosyne, the spaced repetition flashcard program I use which also happens to be written in Python.  It's ten times larger than the previous version, but at 20 megabytes that's not a big deal.  A quick look suggests that this is due to the inclusion of mplayer and the updated version of QT being used.

There was no information I could locate about the repercussions of installing 2.0 over an existing 1.x installation, so I went ahead and installed it under a different menu and installation directory name.  Either as part of the installation process, or when first run, it located my existing cards and imported them.

The revision interface looks pretty much the same.  The card browsing interface which I didn't screenshot, is much improved however.

There's also new support for synchronisation with mobile devices built in, and a whole range of other more standard features used in the implementation of the new version.  I haven't had a chance to explore these yet, but am looking forward to it.

The older version had support for user authored plug-ins written in Python, which I only used to convert Chinese pinyin numbers to tone marks.  Unfortunately, there's no matching support in the new version yet.  For a long time, it seemed like this was one of those projects where the author had given up on it and was effectively distracted by a never-ending "rewrite".  Glad to see it's not the case.

Paint.NET file type plugin development

Paint.NET isn't very well documented in terms of what you need to do to develop a plugin.  And if your plugin is intended to add support for a new file type, rather than add a new image-related effect.. then there's even less information.  Luckily you can download a third-party template and use that as a base.

Ensure the project is set to use the .NET 3.5 framework (mine defaulted to 4.0).  If this is not done correctly, then your plugin will be ignored by Paint.NET.

Copy your compiled DLL from the Visual Studio project build directory to the FileTypes folder within your Paint.NET installation.

And that's pretty much the compile and test (or installation) cycle covered.  Debugging is another matter.  I'm not sure its possible with Visual Studio Express, but I think it is with the full version.  I had to make do with custom message boxes, but it could be worse! :-)

An alternative lossless video codec

I've been wanting to install a lossless video codec so that I can record something or other.  The recommended codec to use is the Techsmith Camtasia screen recording codec.  However, this only decodes and lacks the support for encoding.  Trying to encode gives the following dialog:

Installing the trial of Camtasia Studio unfortunately doesn't magically open up the encoding functionality for the trial period.  Luckily there's a free alternative in the confusingly named CamStudio.  CamStudio provides its own lossless video codec based on both LZO and GZIP.

There are a few teething issues.  I use Windows 7 64 bit and rhe codec doesn't appear in the list offered within the application I want to capture video within.  But thankfully moisamva has provided instructions on how to manually correct the installation, and they work for me.

Roguelike MUD progress #6 - Chinese character sets

Previous post: Roguelike MUD progress #5

Feeling like doing a bit of programming, I decided to run my roguelike MUD code and see if there was anything that I might feel drawn to working on.

This must be the first time I have tested it since I bought a Chinese laptop. I have debug menus in-game which show me all four character sets, and they now show the following.

Windows Telnet looks better than Putty, but if Putty is what I have to use now, then that's the easiest path forward.

It looks like Putty is also not working reliably now. Going back to Windows Telnet, maybe there is some way to get DOS to display CP 437 without having access to the ability to switch operating system languages. And it turns out there is, typing chcp 437 in a DOS console does the trick. But unfortunately, it uses a different font and it also seems to break handling of escape codes.

Unfortunately, this is going to require a lot more research and work. I can't say I am enthused about it, as I would rather be just working on game code. But maybe I can detect Chinese Windows and use unicode characters, as they seem to be available.

Windows command prompt zip support

I've been writing a script to fetch and build a set of dependencies, and trying to keep it simple. But SDL includes the Visual C project and solution files in a zip file. This is a problem because Windows does not have command line zip file support. The point of this exercise is to just get the job done, not to add other new dependencies.

Something in Windows that jumps out when you have this need, is that there is support for compressed folders. In an Explorer window, you can simply double click on a zip archive and treat it like any other folder. Surely there must be a way to access this functionality from the command line..

My first port of call was Powershell. This comes with later versions of Windows, and shouldn't be an unreasonable dependency - surely it has zip file support? No, this is not the case. But it turns out someone used COM object support to access the shell and work with Explorer.  Unfortunately, this was a false lead.  Powershell lives in its own bubble where scripts need to be signed, to be executed behind the scenes as part of some abstract compilation process.  It isn't acceptable for me to say, go into Powershell and turn off this script security setting that requires signing.  That's worse than adding a dependency, it's a strange incomprehensible complication.  Not doing it gives this lovely error, but it probably gives it to you in equally comprehensible English.

At this point I thought, you know, vbscript never had this problem.  So I searched for "wscript", "zip" and assorted other keywords and came across this script which did exactly the same as the Powershell script, but in straightforward and directly usable vbscript.  Further searching of course was a reminder that all the useful answers are available on the Stack Overflow family of sites, in this case Server Fault.

Problem solved. Of course, nothing comes for free. This is an Explorer action, so copying from a zip folder to another folder, will pop up a file transfer dialog with a progress bar for the duration. However, that is a small price to pay for simplicity.