Converting 96kHz 24-bit FLAC to OGG with ffmpeg

Lately my son Robin asked for Peter Gabriel’s song The Tower That Ate People in a car. I like OGGs, although recently it may have been pointless with MP3 patents being expired. But 15+ years ago it was an obvious choice for me, especially because most encoded MP3 files had also clearly cut out high frequencies and generally lower quality at the same bitrate. Again – not a problem I encountered with newer MP3s. But I stayed true to OGG and I honestly don’t need anything better than its Q7 level.

The song is on Peter’s OVO album but the version Robin likes is from Back to Front show in London. So I browsed it, played it and – all the songs were skipped. Darn! I knew it must be because of the quality being very high because the digital download, companion to the Blu-ray Deluxe Book Edition (yeah, I’m a fan), was in 96kHz for both FLAC and OGG. So I had to recode the OGG, or better FLAC to OGG in normal sample rate (44.1kHz).

FFmpeg for the rescue!

I previously transcoded OGGs to MP3 for a little radio that didn’t support OGGs (I never understand why this happens) and I was very satisfied with FFmpeg because when I can do something from a command line I prefer that. So today I downloaded Windows build of FFmpeg and tried to figure out the switches.

After some Googling I tried -codec:a libvorbis and it told me there is no such a codec. So I tried ffmpeg -codecs to find out what (and if) there is any OGG support. There was just vorbis decoder, so I tried that one. Then ffmpeg told me that it’s just experimental and I must add -strict -2 switch to enable it. It worked afterwards but the warning was strange so I investigated further.

The trouble was that the build from FFmpeg site did not have libvorbis compiled in. Every time you run ffmpeg it prints the configuration it was compiled with and mine didn’t show –enable-libvorbis in the output. It was by an accident I found out I’ve got ffmpeg also on my PATH – which was strange considered I didn’t put the downloaded version there. It was part of ImageMagick which I was pretty sure was installed using Chocolatey (most recommended!), I don’t even remember why. But now it came handy, because, behold, this one had libvorbis with it!

If you have Chocolatey already, just cinst -y imagemagick and then start a new console to find ffmpeg on your path. Or do it the hard way.

Those damn spaces!

I use spaces in the filenames, replacing them with underscores or something does not make much sense, not in 21st century I believe. I respect bash (and prefer it in Windows as well, as delivered by Git) and I consider myself more or less a power-user (more less than more I guess). I understand that long time ago text was the thing and objects were not. But all this white-space escaping is sometimes killing me. Just look at all the effort that went into escaping white-spaces – IFS, quoting, print0, etc.

Actually, using NUL character (print0) as a separator seems most logical but obviously it’s difficult to put it into plain text then. But plain text is so awkward to represent anything anyway (except for the actual text). I believe some richer environment where lists are true lists is the logical thing to have. I’m not hinting on PowerShell necessarily, not sure they have it right, but they tried to be progressive for sure.

When I quote the name containing spaces on the input it’s a single argument (let’s say $1). But when I use ffmpeg -i “$1″… in the script the program complains that the first word from the filename is not a valid name. I encountered this problem many times before, passing the arguments from a command line to the script and there to other commands. Today I learned that “${1}” is different from “$1”. I always used curlies only to separate name of a variable from potentially colliding surrounding. But the first one keeps $1 as a single parameter even for another executable called from a script. Handy. Not intuitive. And definitely not something you learn in this section, for instance.

If this was all more “object-oriented” (broader meaning) it would be a filename, String or even File object from the start all the way to where it should be used. Spaces would not matter.

Sample rate and unexpected “video” stream

Because the source flac file had sampling rate of 96kHz – and I suspected this was the main reason the car audio system didn’t play it – I wanted to resample the audio to more traditional CD quality. That’s what option -ar 44100 does. Because OGG seems to have a single sample format, I didn’t have to care about bringing 24bits down to 16.

But I was surprised that my OGG didn’t play in foobar2000 and loading it actually created two entries in a playlist. I checked the output of a command more carefully and noticed it also converted some JPEG image embedded in that FLAC to a “video” stream. Not interested, thank you, said I – and that’s what -vn (no video) switch does.

And the script is…

Add setting the quality of the output OGG and -y to overwrite the output (I experimented repeatedly, you may not want it, of course) and you get a script like this:

#!/bin/sh

ffmpeg.exe -i "${1}" -ar 44100 -vn -codec:a libvorbis -qscale:a 7 -y "${1%flac}ogg"

It only encodes one file. Last thing I wanted is to treat input arguments for a for loop, although I guess I could have used shift too. Anyway, the command is easy:

find . -name \*.flac -exec ./anything-to-ogg-44k1-q7.sh {} \;

I guess it doesn’t care about the input format as long as it recognizes it, hence the “anything”. Of course, ffmpeg can do much more – I just wanted to show one recipe, that’s all.

Advertisements

Building Windows VirtualBox machines

I started this post in January originally, but after a couple of paragraphs I realized I’m writing a more generic post – Believe in build automation. Now you know why I believe in automation and we can get straight to it. I’m a Linux guy, I’d rather work with Linux, I always prefer UNIX/Linux on servers, but I run Windows desktop to be conformant. After all I can run anything in VirtualBox when I need it.

And sometimes what I need is just another Windows. But I don’t want to manually prepare the box all over again after it expires (evaluation), not to mention I want a repeatable process (because I believe in it :-)). You may snapshot your virtual machines, but you cannot avoid eventual end of evaluation period.

State of affairs in Windows automation

Couple of years ago I got a new computer with Windows and I wanted to put all my favourite tools on it. Of course I didn’t have a list. But I had a feeling I’m repeating myself. I also wanted to disable some Windows features. I had experimented with PowerShell before, so I turned to it with faith. I found out that there are some PowerShell modules that allow to add/remove features or applications, but they are limited only to Windows Server. Couple of ugly words ran through my head and I postponed my dream.

Now I know this was a hasty decision, because Microsoft does not offer just one good standard way how to do it. As explained here, you can use one of two PowerShell modules. ServerManager module was the one that made a bit angry because of its restrictions to server versions of Windows, but there is also Dism module available on any recent platform, not to mention dism.exe itself, that works for older Windows incarnations as well.

While this all is just a minor episode, it documents how difficult it may be to find the right way how to perform various tasks on the command line (and preferably PowerShell) for a newcomer. And I wasn’t even that new on Windows.

But after this it was easier and easier to use the right words and ask the right questions the right way to get my answers. Most of them were on StackOverflow, but I have to praise Microsoft’s sites too. Sure, sometimes you have to go over couple of Microsoft pages, but in overall you can find the answers.

Back to Windows virtual machine creation

Here you go, I nearly did it again! Wrote a different post than I wanted, that is. So back to the topic. Of course, you need to know your options for automation, so learning more about PowerShell and about ways how to (un)install various Windows features is still important. But we also need to know the general workflow how to bring Windows virtual machine to life. I decided to use Vagrant because it aims for developers and is praised by them.

When it comes to Windows there is one big trouble – because Windows is big. The same trouble exists for Linux too, but is smaller. We’re talking about automated installation. Good news is that both systems can be installed automatically. It comes as no surprise for Linux, but Windows also features so-called “unattended installation” which aims for corporate world where admins don’t want to sit through installations for all the computers in a big company.

It works in a simple way – you provide an XML file with this unattended configuration for the computer and Windows finds it during installation. It can be found on a floppy or USB drive.

I don’t know all the options how to install Windows in an automatic fashion, but this one is good for virtual machines. All you need to do is provide the booting virtual machine DVD with Windows ISO image and Autounattend.xml file on a virtual floppy disk.

Can Vagrant do it? Ok, now you got me. 🙂 Probably it can, but a brief investigation on the Internet revealed that instead of doing this installation with Vagrant I should first use Packer. And – what a surprise – both tools are developed by the very same company/guy (HashiCorp/Mitchell Hashimoto, sorry for leaving out any other participants from the company). I was not the only one confused about the differences between Vagrant and Packer.

Shortly (and maybe not 100% precise), Packer is good in creating virtual machine base images and Vagrant is good to use them in your development process. Base image is not installation ISO, it’s rather a snapshot of a virtual machine after installation with everything you want to have there for the start. Not with everything possible though, that’s why it’s called base image. Packer can build base images for various virtualization platforms, but we will focus on VirtualBox only.

My idol Matt Wrock

I decided to install the Windows somehow automatically using Vagrant and prefer PowerShell as much as possible. Searching for a solution I somehow found the post called In search of a light weight windows vagrant box by Matt Wrock. I read a bit of it but I was also intrigued by the link to the updated version using Packer and Boxstarter. Already by this time I was overflowed by new terms, but it was worth it, I promise!

Matt definitely knows his stuff, his Windows and automation experience is extensive and he can explain it properly as well. Just a day before that I wasn’t ready to add Packer to my arsenal, not to mention Boxstarter – and soon I learned about Chocolatey as well. Now, honestly, I still don’t get Boxstarter, so for me it’s just “some extension for PowerShell” (disregard at will), but I absolutely fell in love with Chocolatey, because with it the management of programs feels like on Linux.

Matt’s instructions how to use Packer and Boxstarter were pretty cool, he provides Packer files (configurations, or sort of recipes, for Packer, written in JSON format) for Windows Server 2012, Windows Nano (very interesting addition to Microsoft’s arsenal) and Windows 7 (here you need license key, as there is no evaluation ISO, shame). I definitely utilized Windows Server 2012R2, as the server edition always comes handy during development when you want to experiment with a domain controller, etc. But I also wanted packer template for Windows 10 – and I had to create the one myself.

Windows 10 experiments

Actually, the biggest problem with Windows 10 wasn’t the Packer template, but with the Autounattend.xml file. I found some generator, but it didn’t deliver without some errors. I’m still pretty sure that XML is far from flawless, it’s not cleaned up properly and so on – but it works. Diving into every detail in a field that is mostly new for me (the whole world of Windows automation) would probably stop me before I got to the result, so take it as it is, or make it better if you can.

I highly recommend to read that Matt’s article Creating windows base images using Packer and Boxstarter as it is a very good introduction into the whole pipeline. His packer templates also provided a great starting point for more experiments. I also liked the way how he minimized the images by removing many Windows components, defragging the disk, zeroing empty space, etc.

I summed up my experiments in a markdown file and looking at it, it definitely is not perfect and finished. But does it have to be? In a week I played with it I probably installed Windows 10 forty times. Most of these test I commented out the slow parts that were mere optimization steps mentioned above (minimizing the size of the image). I played a lot with some preinstalled software (using Chocolatey, of course) and tried to pre-configure it using registry changes where necessary. This is, however, in vain, as the sysprep.exe step wipes registry changes for vagrant user. Talking about vagrant user, be extra careful to spell it everywhere the same way. Once I messed it up, had vagrant in Autounattend.xml and Vagrant in postunattend.xml (which is used as C:\Windows\Panther\Unattend\unattend.xml by sysprep.exe) and had two Vagrant accounts – you don’t want that. 🙂

I tried hard to perform some installation and configuration steps after sysprep, I tried to change it from Packer’s shutdown_command to windows-restart provisioner step but I wasn’t able to overcome some errors. After a while I settled with a script I just copied to my Vagrant working environment directory and then ran it from initialized box where it appeared in c:\vagrant directory.

Sure I could do even better with full automation, but when things resist too much sometimes it’s good to step back, rethink the strategy and focus on quick wins.

Other options?

There are definitely more ways how to prepare Windows 10 box, or any Windows for that matter, with or without Packer, but even when we focus on Packer solutions there’s a wide spectrum of approaches. Some don’t bother to use sysprep.exe to generalize their installation – after all if it’s only for personal needs it really is not needed. People on GitHub seem to agree on using PowerShell as a Packer provider, but one of the solutions used no provider at all (all part of the build section). Also widen the search for other Windows versions and you’ll see much more variability.

Conclusion

Using Packer as a first step in the pipeline is very practical. You can prepare base image once and save a couple of hours any time you need a fresh environment (partially you can do the same with snapshots in VirtualBox, but it’s not the same).

I use Boxstarter in the process as recommended (although I’m not able to appreciate it fully) and Chocolatey to install/remove programs – during Packer steps and also anytime later. When my evaluation Windows runs out I simply refresh the packer image from ISO and I’m done for the next 90 or 180 days (depending on the OS version).

Following the installation enthusiasm I went on to install SQL Server 2014 Express from Chocolatey package, configure it using PowerShell bits and pieces found across many blogs and stackoverflow questions and wrote it down on GitHub. I actually got used to writing technical notes I may need later into these markdown files and it works very well for me. Now I have automation hiatus, but I’m sure I’ll get back to it and it’s good to find all the notes at hand, including unresolved problems and ideas.

Good luck with Windows automation!

Last three years with software

Long time ago I decided to blog about my technology struggles – mostly with software but also with consumer devices. Don’t know why it happened on Christmas Eve though. Two years later I repeated the format. And here we are three years after that. So the next post can be expected in four years, I guess. Actually, I split this into two – one for software, mostly based on professional experience, and the other one for consumer technology.

Without further ado, let’s dive into this… well… dive, it will be obviously pretty shallow. Let’s skim the stuff I worked with, stuff I like and some I don’t.

Java case – Java 8 (verdict: 5/5)

This time I’m adding my personal rating right into the header – little change from previous post where it was at the end.

I love Java 8. Sure, it’s not Scala or anything even more progressive, but in context of Java philosophy it was a huge leap and especially lambda really changed my life. BTW: Check this interesting Erik Meijer’s talk about category theory and (among other things) how it relates to Java 8 and its method references. Quite fun.

Working with Java 8 for 17 months now, I can’t imagine going back. Not only because of lambda and streams and related details like Map.computeIfAbsent, but also because date and time API, default methods on interfaces and the list could probably go on.

JPA 2.1 (no verdict)

ORM is interesting idea and I can claim around 10 years of experience with it, although the term itself is not always important. But I read books it in my quest to understand it (many programmers don’t bother). The idea is kinda simple, but it has many tweaks – mainly when it comes to relationships. JPA 2.1 as an upgrade is good, I like where things are going, but I like the concept less and less over time.

My biggest gripes are little control over “to-one” loading, which is difficult to make lazy (more like impossible without some nasty tricks) and can result in chain loading even if you are not interested in the related entity at all. I think there is reason why things like JOOQ cropped up (although I personally don’t use it). There are some tricks how to get rid of these problems, but they come at cost. Typically – don’t map these to-one relationships, keep them as foreign key values. You can always fetch the stuff with query.

That leads to the bottom line – be explicit, it pays off. Sure, it doesn’t work universally, but anytime I leaned to the explicit solutions I felt a lot of relief from struggles I went through before.

I don’t rank JPA, because I try to rely on less and less ORM features. JPA is not a bad effort, but it is so Java EE-ish, it does not support modularity and the providers are not easy to change anyway.

Querydsl (5/5)

And when you work with JPA queries a lot, get some help – I can only recommend Querydsl. I’ve been recommending this library for three years now – it never failed me, it never let me down and often it amazed me. This is how criteria API should have looked like.

It has strong metamodel allowing to do crazy things with it. We based kinda universal filtering layer on it, whatever the query is. We even filter queries with joins, even on joined fields. But again – we can do that, because our queries and their joins are not ad-hoc, they are explicit. 🙂 Because you should know your queries, right?

Sure, Querydsl is not perfect, but it is as powerful as JPQL (or limited for that matter) and more expressive than JPA criteria API. Bugs are fixed quickly (personal experience), developers care… what more to ask?

Docker (5/5)

Docker stormed into our lives, for some practically for others at least through the media. We don’t use it that much, because lately I’m bound to Microsoft Windows and SQL Server. But I experimented with it couple of times for development support – we ran Jenkins in the container for instance. And I’m watching it closely because it rocks and will rock. Not sure what I’m talking about? Just watch DockerCon 2015 keynote by Solomon Hykes and friends!

Sure – their new Docker Toolbox accidentally screwed my Git installation, so I’ll rather install Linux on VirtualBox and test docker inside it without polluting my Windows even further. But these are just minor problems in this (r)evolutionary tidal wave. And one just must love the idea of immutable infrastructure – especially when demonstrated by someone like Jérôme Petazzoni (for the merit itself, not that he’s my idol beyond professional scope :-)).

Spring 4 and on (4/5)

I have been aware of the Spring since the dawn of microcontainers – and Spring emerged victorious (sort of). A friend of mine once mentioned how much he was impressed by Rod Johnson’s presentation about Spring many years ago. How structured his talk and speech was – the story about how he disliked all those logs pouring out of your EE application server… and that’s how Spring was born (sort of).

However, my real exposure to Spring started in 2011 – but it was very intense. And again, I read more about it than most of my colleagues. And just like with JPA – the more I read, the less I know, so it seems. Spring is big. And start some typical application and read those logs – and you can see EE of 2010’s (sort of).

That is not that I don’t like Spring, but I guess its authors (and how many they are now) simply can’t see anymore what beast they created over the years. Sure, there is Spring Boot which reflects all the trends now – like don’t deploy into container, but start the container from within, or all of its automagic features, monitoring, clever defaults and so on. But that’s it. More you don’t do, but you better know about it. Or not? Recently I got to one of the newer Uncle Bob’s articles – called Make the Magic go away. And there is undeniably much to it.

Spring developers do their best, but the truth is that many developers just adopt Spring because “it just works”, while they don’t know how and very often it does not (sort of). You actually should know more about it – or at least some basics for that matter – to be really useful. Of course – this magic problem is not only about Spring (or JPA), but these are the leaders of the “it simply works” movement.

But however you look at it, it’s still “enterprise” – and that means complexity. Sometimes essential, but mostly accidental. Well, that’s also part of the Java landscape.

Google Talk (RIP)

And this is for this post’s biggest let down. Google stopped supporting their beautifully simple chat client without any reasonable replacement. Chrome application just doesn’t seem right to me – and it actually genuinely annoys me with it’s chat icon that hangs on the desktop, sometimes over my focused application, I can’t relocate it easily… simply put, it does not behave as normal application. That means it behaves badly.

I switched to pidgin, but there are issues. Pidgin sometimes misses a message in the middle of the talk – that was the biggest surprise. I double checked, when someone asked me something reportedly again, I went to my Gmail account and really saw the message in Chat archive, but not in my client. And if I get messages when offline, nothing notifies me.

I activated the chat in my Gmail after all (against my wishes though), merely to be able to see any missing messages. But sadly, the situation with Google talk/chat (or Hangout, I don’t care) is dire when you expect normal desktop client. 😦

My Windows toolset

Well – now away from Java, we will hop on my typical developer’s Windows desktop. I mentioned some of my favourite tools, some of them couple of times I guess. So let’s do it quickly – bullet style:

  • Just after some “real browser” (my first download on the fresh Windows) I actually download Rapid Environment Editor. Setting Windows environment variables suddenly feels normal again.
  • Git for Windows – even if I didn’t use git itself, just for its bash – it’s worth it…
  • …but I still complement the bash with GnuWin32 packages for whatever is missing…
  • …and run it in better console emulator, recently it’s ConEmu.
  • Notepad2 binary.
  • And the rest like putty, WinSCP, …
  • Also, on Windows 8 and 10 I can’t imagine living without Classic Shell. Windows 10 is a bit better, but their Start menu is simply unusable for me, classic Start menu was so much faster with keyboard!

As an a developer I sport also some other languages and tools, mostly JVM based:

  • Ant, Maven, Gradle… obviously.
  • Groovy, or course, probably the most popular alternative JVM language. Not to mention that groovsh is good REPL until Java 9 arrives (recently delayed beyond 2016).
  • VirtualBox, recently joined by Vagrant and hopefully also something like Chef/Puppet/Ansible. And this leads us to my plans.

Things I want to try

I was always friend of automation. I’ve been using Windows for many years now, but my preference of UNIX tools is obvious. Try to download and spin up virtual machine for Windows and Linux and you’ll see the difference. Linux just works and tools like Vagrant know where to download images, etc.

With Windows people are not even sure how/whether they can publish prepared images (talking about development only, of course), because nobody can really understand the licenses. Microsoft started to offer prepared Windows virtual machines – primarily for web development though, no server class OS (not that I appreciate Windows Server anyway). They even offer Vagrant, but try to download it and run it as is. For me Vagrant refused to connect to the started VirtualBox machine, any reasonable instructions are missing (nothing specific for Vagrant is in the linked instructions), no Vagrantfile is provided… honestly, quite lame work of making my life easier. I still appreciate the virtual machines.

But then there are those expiration periods… I just can’t imagine preferring any Microsoft product/platform for development (and then for production, obviously). The whole culture of automation on Windows is just completely different – read anything from “nonexistent for many” through “very difficult” to “made artificially restricted”. No wonder many Linux people can script and too few Windows guys can. Licensing terms are to be blamed as well. And virtual machine sizes for Windows are also ridiculous – although Microsoft is reportedly trying to do something in this field to offer reasonably small base image for containerization.

Anyway, back to the topic. Automation is what I want to try to improve. I’m still doing it anyway, but recently the progress is not that good I wished it to be. I fell behind with Gradle, I didn’t use Docker as much as I’d like to, etc. Well – but life is not work only, is it? 😉

Conclusion

Good thing is there are many tools available for Windows that make developer’s (and former Linux user’s) life so much easier. And if you look at Java and its whole ecosystem, it seems to be alive and kicking – so everything seems good on this front as well.

Maybe you ask: “What does 5/5 mean anyway?” Is it perfect? Well, probably not, but at least it means I’m satisfied – happy even! Without happiness it’s not 5, right?

Making Windows 7/8 work again

OK, kinda questionable title, as Windows somehow works after install (and reinstall). But I have some additional professional needs. I don’t want to edit system properties in a single line, I want proper command line – preferably bash – and when at it, I’d welcome many Unix text/file utils too. I want proper console that resizes, allows more than silly block-mode copy. I want alt-tab working properly and not disappearing after a while. Better basic editor.

I wrote about some of the stuff, but maybe someone will find it helpful concentrated in a single post.

I’m actually no tools freak, I don’t check the landscape for new fancy tools every now and then, so there may be better alternative (“better” can have many meanings though :-)). So let’s just go through the stuff I need to do with a fresh Windows installation. Some stuff is related to 7 or 8, but most applies for both.

Many of my needs come from my profession. I’ll omit Java, Gradle/Maven, TortoiseSVN and others, but I’ll name those that help me be productive in command line for instance.

Alt Tab working again (Win 7)

Depends whether you like it or not. I want Alt+Tab to display the application switcher and keep it that way. Because that’s how I switch faster. Aero Peek is function that displays the application – which is cool when you’re in doubt which of those many same icons is the one you want – but when you continue hitting Alt+Tab it just does not display the switcher anymore. That sucks. Hit the Start button (or WinKey) and write “performance”. One of the filtered options should be “Adjust the appearance and performance of Windows”. Go for that. In Visual Effects tab, there is long list of checkboxes. And you want to disable Enable Aero Peek. Never missed it actually.

Windows 8 adjustments

I’ll be swift here, as I have adjusted just a single installation so far. My biggest concern I remember – from those that are easy to fix – was the sticky corner (corner clip) when I used dual-head. When I move window from one monitor to the other, I slide on the top edge – and I stumble upon the edge of the screen here. Fix is easy, although you have to edit the registry.

And then there was Narrator on Win+Enter which I accidentally pressed here and there. Just delete the executable or disable it other way. It can’t be disabled in any normal way.

Browsers, more browsers

So this one is obvious. What you need Internet Explorer for? To download Firefox (Chrome, whatever), right? Then you may need some Web Developer extension, Firebug and you’re ready to go.

BTW: If for nothing else, then for funny bug that ruins my Google Drive document writing on Firefox every time… copy/paste works for a while and then suddenly it stops. Restart does not help, nothing seems to help. Chrome does not have this problem (or it didn’t manifest so far). Copy/paste on Google Drive is quite popular problem so it seems, I don’t care who’s “fault” it is here. Just one of those many funny things about current innovations. 😉

Notepad2 – what Notepad was supposed to be

No tabs, no complications, just Notepad. But with toolbar (should you want it) and some basic settings options. Better status line. That’s Notepad2. My biggest gripes with default Notepad are:

  • doesn’t work with unix newlines properly;
  • no reasonable support for various file encodings or re-coding.

Notepad2 does have these and some basic syntax highlight, newline conversion, regex search and more. Check FAQs to find out Notepad2’s position on some additional features. If it had hex editing, I’d probably never need anything else. For anything serious I use IntelliJ IDEA anyway.

Do you need a bigger gun? Go for Notepad++, Ultraedit, or whatever else, the list is way too long. What I like about Notepad2 is that it is exactly what I’d expect from Notepad – simple single document editor that doesn’t lack features you need way too often today – especially work with various encodings. Instead of reconfiguring apps to use this as a default editor, or associating file extensions, I simply replace both c:\windows\notepad.exe and c:\windows\system32\notepad.exe with this one. You need to claim the ownership and all this hardness, but it is well worth it.

Or you may try this BAT to do it (not sure how it works with domains, but you can’t screw anything except those notepad.exe binaries :-)). It uses notepad2.exe already copied into Windows directory which you have to do as Administrator:

REM run as admin
c:
cd \windows

takeown /f notepad.exe
icacls notepad.exe /grant %USERNAME%:F
copy notepad.exe notepad.exe.orig
copy notepad2.exe notepad.exe

takeown /f system32\notepad.exe
icacls system32\notepad.exe /grant %USERNAME%:F
copy system32\notepad.exe system32\notepad.exe.orig
copy notepad2.exe system32\notepad.exe

Total Commander

Or any other beast you like. I personally can’t get rid of these two-panel commanders. Bookmarks, tabs, many handy features (synchronize, multi-rename tool, built-in ZIP). I strip my Total Commander to hide F-buttons, toolbar, command line (right-arrow displays it) switch Quick Search to Letter only – and I’m ready to go.

Git

Ok, so I excluded SVN but here I come with Git – what is that? Well with Git for Windows you’ll get git bash – that is bash! And tons of useful Unix commands, though not all of course. So for the rest I go for…

GnuWin32

GnuWin32 contains tons of GNU tools ported for Windows. I don’t know the status of GnuWin64, but GnuWin32 works just fine on 64-bit system too. All packages neatly put their binaries into common bin directory, so a single PATH entry is enough.

Give some thought to your PATH order, I personally have git’s binaries before GnuWin32 because they are in some cases better integrated with its bash. But if you use also GnuPG, put it even before git.

Rapid Environment Editor

Talking about variables… I mentioned this one already some time ago. This is my hero for setting environment variables. Download it, use it, love it. It is handy to put all the *_HOME variables (JAVA_HOME, GRADLE_HOME, …) to System Variables and just reference them in PATH. And for bash usage it is handy to define HOME (for cd without arguments if for nothing else) in User Variables and set it to %USERPROFILE% – use variable of type Expandable String for that. Don’t forget to Save it (Ctrl+S as expected) and restart any program you expect using the stuff.

Real console, please!

And here I’m split a bit. I used Console2 for a long time, works fine, except it doesn’t maximize in Windows fashion. I also used ConEmu on another computer. Works fine, maximizes… but I didn’t work with it that hard like with Console 2. So I don’t know. ConEmu seems to have more features and more actively developed, so I’d check that one now. Both are far better than default cmd, where you – of course – can switch to bash, when you have it on path. For ConEmu, download something that extracts 7z – like 7-Zip – unless you go for MSI (whatever works better for you) or you install a packer plugin (WCX) to your Total Commander.

As for ConEmu settings (Win+Alt+P), you may want to make some changes:

  • First in Keys & Macro – Controls section disable Change prompt text cursor position, so we can make copy/paste work both from left mouse button select and from other programs too.
  • In Keys & Macro – Mark/Copy section for Text selection change the value to Always.
  • In Keys & Macro you may want to change Scroll buffer one page up/down from Ctrl+PgUp/Dn to Shift. It may be easier to find the functions if you sort the list by Description.

With these two I’m mostly ready.

Console with bash from Total Commander

BTW: I talked about Total Commander and Git Bash Here integration already – using Console 2. For ConEmu, let’s do it with editing your INI files – they will probably be in %USERPROFILE%\AppData\Roaming\GHISLER\. Just define your custom command in usercmd.ini (adjust paths of course, bash is in PATH already in my case):

[em_bash]
button=C:\work\tools\ConEmu\ConEmu.exe
cmd=C:\work\tools\ConEmu\ConEmu.exe
param=-cmd bash --login -i

And in wincmd.ini just use this command – I chose Ctrl+B shortcut:

[Shortcuts]
C+B=em_bash

Happy bashing.

Bashing who?

Obviously I’m no big Windows lover. I even hate it as a server platform. I love ssh to my boxes spitting commands on them. This is probably possible for Windows too, but actually not many people use it. I’m still waiting to meet good Windows administrator and enthusiast who will show me some meaning of Windows as a server. Combined with Java. Screw performance comparisons now, just talking about my life as an administrator or developer.

But I’m using Windows in my daily job as my desktop and I like many of the stuff. I don’t turn it into Linux (ok, bash, GnuWin…, but…), I don’t use virtual desktops (I loved them), or any advanced keyboard shortcut solutions. All these tools and adjustments are really rather elementary to me.

I like some of the Windows stuff (when I remove some other). So I don’t want to bash Windows. Installing and integrating all the things together is no doubt more demanding than a couple of apt-get commands Linux users are used to. But then, they often need to go through many configurations as well.

Cool stuff that works

Learn WinKey shortcuts. It really pays off. You may stun your boss when you tell them, “just press Win+Shift+Right to get it on the projector” (switching app between screens).

Pin your applications on the taskbar and start or switch to them using Win+number.

If you use localized keyboard a lot, learn AltGr shortcuts, at least the basic ones. When you need a single curly brace it is much faster to press AltGr+B than Alt+Shift, { and Alt+Shift again. Often these AltGr symbols are printed on the keyboard.

Windows 8?

Windows 8 is generally step in a wrong direction for (not only) me, but it has some interesting additions. While you can’t Alt+F4 after closing the last application to get Shutdown dialog anymore, you can use new Win+X (Power User Menu) and then just double tap U, U to do the same. And many, many more through this menu. There is more for sure, but generally Windows 8 rather does maximum to get into our way if we use the PC the old way (no touch display, etc.). Kinda shame.

I hope you found some of the tips and tools helpful. If you have your tricks and favourite tools, feel free to share them in comments! 🙂

Releasing to Maven Central with Git on Windows

Originally I wanted to write about two problems. I think both were somehow related to interactions between Maven, Git and GPG on my Windows. But because I didn’t document it both on time, things got lost. So this will be only half of the story.

Short version? Don’t mix various gpg on your PATH. If you use Gpg4Win, prefer that one.

Switching to Git

When you start with Git, you’ll probably get lost often even when you get the basic ideas and read those intro manuals. There was no big deal about SVN coming from CSV, but Git is different beast altogether and definitely worth studying. For example, it may happen that your local master branch is ahead of remote master by hundreds of commits (the whole history really), while push is pushing just new commits… And you have no idea what you did first – and I bet there will be more surprises like this. (Little note, for me fetch didn’t work, but the second answer with rebase -p did. Don’t ask, I still have to study it. :-))

With just a few glitches like this I was reasonably afraid of my first release to Maven central with Java Simon on Git repository. But it was much smoother in the end, especially from Maven’s side. Credit to Maven this time, I remember its release plugin throwing bugs at me – and now it did its job with VCS I don’t understand yet.

GPG vs Git’s GPG

The problem I managed to write down is related to the fact, that I had Gpg4Win installed previously (for Maven Central requires signed artifacts) and Git brought another GPG with it too. So this is – obviously – kinda Windows specific.

First I needed to create my new GPG keys for signing artifacts going to Maven central, because for unknown reasons I elegantly deleted the .gnupg directory some time before. So you create the key with:

gpg --gen-key

And then you go to upload it:

gpg --keyserver hkp://pool.sks-keyservers.net --send-keys 08XXXXXX

And there is this error:

gpg: sending key 081A513E to hkp server pool.sks-keyservers.net
gpg: system error while calling external program: No error
gpg: WARNING: unable to remove tempfile (out) `...\AppData\Local\Temp\gpg-515B3D\tempout.txt': No such file or directory
gpg: no handler for keyserver scheme `hkp'
gpg: keyserver send failed: keyserver error

It sounds like anything but the proper solution – your gpg did not find gpgkeys_* executables in the same directory. You can eventually Google it and there is some StackOverflow question too, but answers are various. This is the first strike of git-bash and my PATH settings. There is /bin/gpg, but you better use gpg from where it is installed:

/c/Program\ Files/GNU/GnuPG/gpg --keyserver hkp://pool.sks-keyservers.net \
--send-keys 08XXXXXX

Fixing the PATH

Cleanest solution is to prefer Gpg4Win, so just put C:\Program Files\GNU\GnuPG to the first spot of your PATH – or at least anywhere before Git’s bin directory (which is also high on my list to prefer its commands before binaries from GnuWin32). If you have C:\Program Files\GNU\GnuPG\pub on the list too, bump it up as well. Or just check what Gpg4Win put onto your PATH and move that. Sorry for being a bit vague here – this happened to me with gpg.exe in GnuPG directory first, but now I have only gpg2.exe there and gpg.exe in pub sub-directory which was added to the PATH. This seems to be a change since 2.x versions of gpg or Gpg4Win.

You could – of course – just delete or rename /bin/gpg.exe. But after unplanned Git reinstall it would be there again. Proper order in PATH is important anyway, although I too feel much better when there are no colliding binaries.

GPG speaking different languages?

Is your GPG talking to you in native language and you want it English? Simply set LC_MESSAGES=C (Rapid Environment Editor recommended, use User Variables). Alternatively just delete GnuPG/share/locale directory 🙂 seems not to hurt anything.

Conclusion

I believe my other problem was also related to gpg, it was another twist on the PATH problem when running Maven – which uses gpg to sign those artefacts with keys you prepared. So the main advice still applies. Organize your PATH properly, preferably in Windows System Variables, as Git’s bash will translate it into UNIX format just fine. Also with proper system-wide PATH things will run properly in both cmd and bash, unless they are shell specific. For instance mvn clean release should work in both.

Git Bash strikes back? (edit 2014-10-05)

Fixing the path is nice, but not working – unless you run bash without git’s initialization. Try running just bash and check your path with echo $PATH. Then do the same with git bash (the difference is bash –login -i instead of just plain bash command). This sources not only .bashrc from your home, but also your .profile and (and here comes the blow) also c:\Program Files (x86)\Git\etc\profile (or similar depending on your installation). This is the file that does git bash from just plain bash, sets the prompt, etc. But it also adds couple of paths at the head of your original path – namely $HOME/bin, . (current dir), /usr/local/bin, /mingw/bin and /bin (three out of these 5 items don’t even exist in my case).

So yes, this actually renders my previous advice useless – but only for git bash! If you’re using cmd/bash you don’t need to change anything and you still have all your needs on your path in the proper order.

And what can you do about it in git bash? Fixing that global profile seems kinda crude, so you may prefix the PATH in your .profile with GnuPG path (again). Or delete that gpg.exe in git installation (talking about crude measures :-)).

Personal Log (2): Leaving Linux, buying guitars

Not that I write continuation of the personal log every time Federer wins a Slam title, but after dramatic final of the Wimbledon it really happened. (Just another instant classic after last year final? :-)) Fifteen and counting so it seems.

I wanted to confess that I stopped using Linux on my computers. I still have out of date Ubuntu on my desktop, but I can’t remember last time I used it. I went for Windows XP after all those Linux years, because I play more games now than ever. Video and music processing is also better supported on Windows – more software to choose from, VST support, etc. Seriously, why to use Wine just to pretend I’m not using Windows, right? Of course – anytime I have to work in command line in Windows, it’s real pain in the ass. When I changed my old computer at work for Lenovo T500 (with some troubles though) I decided to go for Windows XP only and in order to have some Linux I searched for some virtual machine solution. Friend of mine recommended VirtualBox and I was satisfied! It worked nicely, there was some trick with virtual folder name (try different name then the directory name if you have problem, or something like that ;-)), but after just a few hours I had nice Ubuntu machine for development tasks that required Unix tools, strong command line and all the rest you don’t have in Windows for free. Of course, performance was… virtual. 🙂 But if you want to try some virtual machine software, I strongly recommend this one. Documentation is OK, not too much to drown in and I found everything important there.

What I can’t understand is SVN Tortoise client. It’s absolutely great tool, looks nice, integrates to contextual menus for files, etc. So far so good. But when you want to check modified files to find out what is not added to repository (and should be) or when you want to commit, the view shows you either only managed files or all files recursively – there is no way that non-managed directory is just reported without the complete sub-tree. Compare this to default CLI functionality of the svn st command. Man I miss that so much! But not that much that I plan to setup my system PATH and run the all-time-loser “cmd” to use it.

There is one more thing I dislike on Windows. I alt-tab a lot, I’m of those erratic guys. 🙂 And sometimes I alt-shift-tab back too. Sometimes, when I depress these keys in a wrong order, Windows is strongly convinced I wanted to switch the keyboard layout (alt-shift). Trouble is, there is limited amount of options (keyboard shortcuts) to do so, this one is clearly the best. I should probably download some tool just do switch between keyboard layouts. Slovak keyboard is not good for coding really. Enough Windows rant for now.

I bought a new guitar! I have infamous Behringer’s V-Tone pack – because it was cheap and I was young(er) and uninformed when I bought it. The price was good, amplifier is good, guitar is… well, it’s some kind of Strat and it is possible to play on it. Now I know there are better options in the same price tag (not too many though), but what’s done is done. To have at least semi-decent guitar I bought French LAG AM100:


Black beauty!

It looks cool, it’s Superstrat, it is something different than the Strat, so I might actually also learn more about guitars per se. It was priced 356€ but in a massive sale action I acquired it for 156€, why not? I actually didn’t know what to buy as I also wanted some Ovation, but I decided to buy the best value from whatever I wanted. Funny sale strategy when you learn prices only on site. 🙂 Even before I bought it I had started to learn some scales and other stuff to enhance my play. Chords, rhythm, no problem, but I wanted to go on after a long stagnation – and with better guitar it’s even more fun!

Is the guitar good? More people claim it’s a good value even for the original price, so I guess I wasn’t wrong with my buy. Quality is OK, but I honestly can’t judge guitar when I can’t compare it. Maybe later I’ll write more on the guitar, trouble with LAG is that they have no official English site (as a manufacturer, not as reseller), forums are dominated by French language, no real documentation for newbies like me. So I have to rely on other forums, other people and learn the rest by myself. Of course, when I just play on it, there is no problem, but you know… when you have a new toy, you want to play with it even more. Tune it (later probably, and I don’t mean just strings now), understand it. However, enough for this time – I need to practice!