Speech impediment - Locale in R under OSX

My R-installation developed a language problem ("L" is a statistical ploglamming language). The startup message, all errors and system messages started appearing in German. I do speak German but it was still quite annoying. This ended with a work-around that I published previously in a post about R-startup scripts. I have finally figured out what causes the error. My Language & Text settings for language were as follows:

1. British English 
2. Swedish
3. Norwegian
4. Danish
5. German 
6. French 

The problem lies in using the British English locale. I use it because I hate when all programs always correct my excellent spelling from Proper English™ to American. R does not have a locale for British English, nor for Swedish, Norwegian or Danish. So, to R, the locale priority list actually looks like this:

1.  
2. 
   
3. 
   
4. 
   
5. German 
6. French

So, understandably it goes for what it knows. The solution is to extend the list and put a standard English locale high up:

1. British English
2. English 
3. Swedish
4. Norwegian
5. Danish
6. German 
7. French 

Using the OS X clipboard in R

One annoyance with using a command line based statistical system, i.e. R, is that if you use a spreadsheet program to handle your data (as most of you do) you have to incessantly export to csv and import to R. For big data sheets and complex calculations this generally a Good Idea(tm). It improves reproducibility and readability when going back to the project at some later time. However, the complexity of that is enough that you often, at least I, end up doing statistics in Excel. This is generally a Bad Idea, and often entails much more work in the long run. In order to make importing and exporting data easier R can handle the clipboard directly, which is what this post is about.

In OS X the clipboard is easily accessed through the command line programs pbcopy and pbpaste (pb means "pasteboard" but everyone knows it as the clipboard so that's what I'm going to call it). These are easily scriptable programs that can be used with any unix command in the shell or scripts (enter "man pbpaste" in the terminal to learn more). "pbpaste" pastes the current clipboard to std-out (that is where you call it from), while "pbcopy" copies std-in (what you put there) to the clipboard. An important quirk is that they only handle plain text, RTF and EPS, that is formats that are transferred as plain text. This is because unix is a text based system. What it means is that you can't copy stuff from formatted text, e.g. word, but with spreadsheets you are OK.

As command line programs they can be handled through the R command: pipe(). Pipe does not produce the current clipboard contents, just a description of the pipe as such. You have to use a read command to get the actual contents.

> readLines(pipe("pbpaste"))

which will produce one string per line (completely useless as data, but often useful to see how a given copy is formatted). To format it for data you use the same as you would any import, using one of the higher level read-commands: read.table(), read.csv(), etc.

> x <- read.table(pipe("pbpaste"))

This will put your copied excel table directly into the table "x" in R. Often you will be forced to tweak the function a little to get a usable table.

> x <- read.table(pipe("pbpaste"), header=TRUE, sep="\t")

That should be able to read your copied Excel data ("header" specifies that the first row is the header, and "sep" specifies the separator which defaults to tab or "\t" in excel).

To write to the clipboard you use "pbcopy". It is a bit more involved, but not much. First you connect a variable to the pipe from pbcopy.

> osxclipboard <- pipe("pbcopy", "w")

Now you can write to the variable "osxclipboard" and it will turn up on the clipboard so that you can paste it into another application. You have to remember that it is a pipe to an external file, which means that you have to use "write()" to put data in the clipboard. If you just assign some data to "osxclipboard" it will be changed from a pipe to that data. Thus:

> write(x, file="osxclipboard")

where "x" is your data, will to the trick. Then you just have to clean up using:

> close(osxclipboard)

In the same way as with copying from the clipboard, this will not format your data optimally. so you have to add some formatting command for Excel to read it properly.

> write(x, file="osxclipboard", sep = "\t")

There we are, but that is rather alot of code. Not really that much easier than a regular export from excel and then import to R. So here are two simple functions to do it all at once, with the most used arguments (I'm sorry for the code, I haven't figured out how to get indenting to work properly in blogger).

Simple readlines from the clipboard:

clipboard.readlines <- function (

clipboardname = "pbpaste",

n = -1L) {

readLines(con = pipe(clipboardname),

n = n)}

Read table from the clipboard. I have only included the most commonly used arguments to read.table(). You could easily add all the rest if you need them:

clipboard.readtable <- function (

clipboardname = "pbpaste",

header = FALSE,

sep = "",

quote = "\"'",

dec = ".") {

read.table(file = pipe(clipboardname),

header = header,

sep = sep,

quote = quote,

dec = dec)}

Write to the clipboard:

clipboard.write <- function (x,

clipboardname = "pbcopy",

sep = " ") {

osxclipboard <- pipe(clipboardname, "w")

write(x = x,

file = clipboard,

sep = sep)

Close(osxclipboard)}

That was it for now. I hope someone finds it helpful. I'm probably not putting any of this into a formal extension, so you will just have to run the code to use it. Consider the code GPL-licensed, although the text is under regular copyright.

Return to the Dark Side

In 1984 my father brought home our first computer. From the year you should be able to guess what it was. In '89 my most used system changed to the new Good Guys(TM): MS-dos, later know as the Bad Guys(TM): M$-Windoze. Being able to play better games was the main reason for that change. Later, 1995 or so a friend brought some disks he had got of a BBS with something called GNU/Linux. For the next ten years I used Linux almost exclusively, including my early years at university and in my research. However, it eventually became apparent that you cannot do research on Linux alone. You need Microsoft Word, and you need EndNote. In addition, you would rather like to have Adobe Photoshop and Illustrator as well.

For years I either double-booted with both MS-Windows and Linux, or, had several computers in parallel. The idea was to have one for heavy work, like statistics and dual-boot it for Windows-programs like Word and Photoshop. Then another computer for most other things. This other computer would run Linux only and be quite small, a netbook, as they later became known as. The result was that I had to bring both computers most of the time. That was a good idea that turned out horribly wrong.

I didn't consider Apple for years, it was expensive, didn't run games and wasn't GNU (If you don't know why that matters, just ignore it). Then I bought a iPhone 4, upgrading directly from a phone with buttons that could send SMS and make phone-calls (Yes, I am a late adopter). And it was slick. As slick as I remembered thinking the Apple Macintosh Plus was in its time. It was easier to check mail with the iPhone than to use the computer. One thing led to another, and yesterday my 13 inch MacBook Pro arrived.

That's enough personal history for one post. The rest is the mandatory setup-story: How to migrate from Linux/Windows dual-boot with Linux-on-the-side to OSX Snow Leopard.

All my old files were easily copied by way of a couple of 1TB USB hard-drives. The hard part was consolidating the three different versions of the project-folders from different computers. For file-organization I opted to keep my age-old folder hierarchy and using the Desktop-folder as root so that I get easily accessible icons. Basically, I have a "job" folder for work-related stuff, and a "div"-folder for other stuff. I then linked the Pictures, Download, Documents, Public and Dropbox folders to the desktop as well. In bash and emacs it makes for a longer path (/Users/user/Desktop/job vs. /home/user/job), but otherwise I think it will be easier.

The Apple backup-facility: "Time Machine" is a dream to setup. It needs no explanation, just use it.

Email was easily imported as mbox-files and, I guess, most other formats too. It automatically handles gmail, appropriately compared to other accounts. It is well built and apparently handles my 15000+ mail-archive without any particular complaints.

Syncing with my iPhone got all my contacts and calendars automagically transferred. Brilliant.

Adobe programs are in essence designed for aApple so they pose no problems, so far. I got CS5 Design Premium to get Illustrator, Photoshop and Bridge (the rest are a bonus). Note that image analysis functionality only is included in Photoshop extended, thus Premium.

MS Word, Powerpoint and Excel are necessary. The Apple variants Pages, Keynote and Numbers are probably nice, but from experience the compatibility is not good enough. I got Microsoft Office 2011.

Then the problems started. EndNote X4, does not work with Word 2011 under OSX. You have to update to 4.0.2, and activate Cite-While-You-Write again in EndNote "Customizations". This may be enough but you might have to copy the "EndNote CWYW Word 2011.bundle"-subdirectory from the CWYW subdirectory to the Word Startup directory in the Office directory. And, then you have to activate the Startup directory for Word under "Preferences - File locations".

(If you didn't know what GNU was earlier, then you can stop reading now. The rest is qualified nerdery.)

One fantastic thing, which I hadn't realized, is that OSX uses proper movement-bindings by default. C-f, C-b, C-n, C-p, C-a, C-e etcetera move the cursor as they should. As usual the Microsoft applications do not conform by default, but it can be fixed. In Word open "Tools - Customize Keyboard" and look for EndOfLine, StartOfLine, LineUp, LineDown, CharLeft, CharRight and EditClear. You have to look under "All Commands", but you can type the first few letters to search. Then just add the proper bindings. Even C-k can be made to work by selecting "Tools - Macros" and creating a new macro with the lines:

Selection.EndKey Unit:=wdLine, Extend:=wdExtend

Selection.Cut

Put them just before the "End Sub" line. Then you go back to Customize Keyboard and set your key-binding.

Emacs is always a bit of a chore to install on any new system. You have to get your bindings just so, and get all your local enhancements to work as they always have (emacs-users are conservative that way, not like the insanely fundamentalist vi-users, but still). The first thing to do is always (setq inhibit-splash-screen t). The splash-screen is a Bad Thing(TM). Then the best way is to use the Apple command-key as Meta since the Macintosh uses both Alt-keys for character-bindings, i.e.

(setq mac-option-key-is-meta nil)

(setq mac-command-key-is-meta t)

(setq mac-command-modifier 'meta)

(setq mac-option-modifier nil)

Today OSX (and most good systems) use UTF-8 as the default character coding system. You can make sure that this works by setting

(set-terminal-coding-system 'utf-8)

(set-keyboard-coding-system 'utf-8)

(prefer-coding-system 'utf-8)

I found the easiest way to install packages was to just place them in my user's site-lisp directory and use load-path for each one. The user site-lisp directory is: "~/.emacs.d/site-lisp/". Then I pretty much copy-pasted my old .emacs and everything seems to be working splendidly.

Caps-lock is Evil, the button to the left of "A" should be a control-key. Luckily, you can change your modifier keys under "System preferences - Keyboard - Modifier Keys".

I end with a short note of thanks to those who wrote the web-pages and manuals where I found many of these hints. I'm sorry to say I didn't save the references while I did the setup, so I have no idea where I stole the ideas from.

Computer down, computer gone.

Shite, the computer to the powerlab went straight to hell during an experiment on Thursday. I thought I saved the day when I ran and got my main laptop. It's got the software and all, but windows update has replaced the drivers with something it likes better. So, while it can start up Chart, and read files, and do analysis, it can't communicate with the powerlab. All in all a really bad day.

So now I'm at home and have just made some nice photos to show you, and the computer is still in the lab. All I have here is my little netbook, and it doesn't have Lightroom. Sorry about that.

There are some other nice blogs that actually have content today, or recently anyway.

There's Isis with Your graduate school application as seen by Dr. Buttercup.

Then we have Ethan at Starts with a bang who is showing a hilarious t-shirt reading:

"Sorry, you have reached an imaginary number. If you require a real number, please turn your phone by ninety degrees and try again."

Finally we have PhDamned (aren't we all) who is overcommitted, which makes me happy, because it means I am not all alone in that predicament.