This is my, well, semi-blog. There is currently no way of adding comments, which can be only sent to me via email (Denny and I will address this remaining limitation when we find the time). You can also obtain these notes as an RSS feed.
I have just re-installed Kubuntu 14.04 64bit from scratch on my Macbook Pro 7,1 (2010). This was surprisingly hard, given that I had been running the exact same system in 32bit mode before. Moreover, it turns out that the problems are not at all related to 64bit vs. 32bit (or to Kubuntu vs. Ubuntu) but to seemingly unimportant details of how you install. It took me almost two days to figure this out, so here is my quick installation guide. Read more …
Hardware: Macbook Pro 7,1. Broadcom BCM432b 802.11 (Wifi). NVidia GeForce 320M (video).
Operating system: Kubuntu 14.04.01 (August 2014), using proprietary NVidia driver 331 and proprietary Boardcom Wifi driver, as suggested by Ubuntu.
I run this as a single-boot system (no MacOS or other OS), but posts on the Web suggest that the problems are the same for dual-boot setups.
Things you MUST know
Most things work very well by following standard (K)Ubuntu installation guides. Drivers for your hardware are available and suggested by (K)Ubuntu. All of my major system functions work perfectly (suspend to ram, wifi, brightness control, sound, touchpad).
However if you install "the wrong way", it will be impossible to get the graphics to work, you will be stuck with the default Nouveau driver (which freezes the machine randomly, does not support brightness changes, overheats the machine, uses up a lot of battery power, and prevents suspend from working), and it will be very hard to repair this state short of installing everything again.
The one important insight is as follows. When installing (K)Ubuntu on this Macbook (and maybe on others):
- Do not install from a USB stick.
- Install from a CD or DVD that you boot up like "Windows" (known as "CSM mode," "BIOS mode," "BIOS compatibility mode," or "legacy mode").
If you know how to do this, then just go ahead and look below for final details (brightness controls etc.). If not, read the next section for details.
BIOS vs. EFI: What, why, how
Today, a computer can boot up in several ways. In the past, all PCs used BIOS and Macs used EFI. Now PCs are switching to a newer, standardised version of EFI, called UEFI, that is not compatible with Mac's custom EFI. Ubuntu (and Linux in general) supports two modes to boot up: "efi" (used for EFI and UEFI) and "pc" (for BIOS and BIOS-compatibility mode). Both work equally well and you won't notice much of a difference in practice.
What's the problem with EFI on this Macbook?
Unfortunately, the proprietary NVidia driver will fail when the machine has booted via EFI. This is official but very badly documented. Moreover, since the error is deeply rooted in things that happen even before the OS is started, the errors you will eventually see are not informative and do not give you any hint at what the real problem is.
If you have installed Ubuntu to boot in EFI mode and are using the (suggested) NVidia drivers instead of the (somewhat flaky) Nouveau drivers, you will see the following issues:
- The boot splash of (K)ubuntu is followed by a black screen (completely black, no backlight)
- The machine does not react to direct inputs at this point (you cannot switch to any tty), but you can still connect via SSH if you installed openssh-server before).
- To use your machine again, you can enter the boot menu to boot into recovery mode, drop to a root shell, and uninstall the nvidia driver.
- After the failed boot, your file /var/log/Xorg.0.log contains no errors, but its last line is something like
[ 6.979] (**) NVIDIA(0): Enabling 2D acceleration
- After the failed boot, you can find an error in your /var/log/kern.log:
kernel: BUG: unable to handle kernel paging request at 00000000000026c0 kernel: IP: [<ffffffffa085ae76>] _nv006050rm+0x27c/0x3bd [nvidia] kernel: PGD 13257d067 PUD 13970f067 PMD 0 kernel: Oops: 0000 [#1] PREEMPT SMP
- followed by a call trace that features several hints to nvidia.
- Adding kernel parameters to boot will not help. I have tried "acpi=off" (seems to get further into boot, even shows a login screen, but this is just because NVidia can't even find the graphics card and Ubuntu falls back to nouveau which then fails later); "noapic nolapic" and "nomodeset" and "xforcevesa" (no effect); and several combinations ("acpi=off noapic nolapic" leads to a different error). The problem is also unrelated to IOMMU incompatibilities of some versions of the Nvidia driver, so "amd_iommu=off" does not help either.
- Using different kernel versions and NVidia driver versions won't help. I have tried many NVidia drivers from 304 to 340, the recommended 331 Ubuntu package, and the manually downloaded version from NVidia. I have also tried the 3.14 kernel as well as some much older kernel which was made for 12.04. No effect in either case.
How to prevent this
Kubuntu will not ask if you want EFI or BIOS mode during installation. It will just use the same mode that you have used to boot the installer. Once you installed Ubuntu in this mode, it will be hard to change. In fact, both modes have their own dedicated boot loaders that work differently (package "grub-pc" for BIOS and package "grub-efi-amd64" for EFI).
How to boot in BIOS mode. Thus, you must make sure that your installer boots in the right mode. For me, this worked by booting from a CD/DVD. My Mac boot loader I installed ages ago (refit or something similar?) allows me to choose a boot medium when holding down ALT during startup. The label "Windows" is used for BIOS mode (whether Windows is involved or not) and the label "EFI" for EFI-mode. If I insert a CD/DVD to boot from, I usually can see two CD icons, of which one says "Windows". This is what I use.
If your boot loader works differently, then you may need to do something else.
At least in my boot loader, it seems to be impossible to boot from a non-EFI USB stick. I could use USB sticks to boot, mainly to use a live system to try and fix my installation, but these were always EFI USB sticks that were then booted in EFI mode.
For the same reason, it might be worth trying with the standard amd64 images provided by Ubuntu, rather than with the amd64+mac images (not sure though; Kubuntu does not have this distinction).
How can you check which mode you have booted in? When vanilla Ubuntu boots in EFI mode, it will have a directory /sys/firmware/efi after boot; otherwise there is no such directory under /sys/firmware/. You can check this even when installing from a Live CD (even without selecting "Try Ubuntu" first): just use CTRL+ALT+F1 (=CTRL+ALT+Fn+F1 on Mac) to go to a tty and type ls /sys/firmware/ to list all entries in this directory; then go back to the graphical UI via ALT+F7.
You will also notice that BIOS booting has some extra delay when starting the machine.
Possible problems getting into BIOS mode
In my case, it was enough to install after booting in BIOS mode. The resulting installation of Kubuntu is started in BIOS mode by the main boot loader as well (since it has no EFI data it could use). I also think that in my boot loader one can "force" BIOS mode as follows: holding down the ALT key at startup in my case always displays a "Windows" icon (even at the times when I have had an EFI-based Ubuntu), and booting this gets you into BIOS mode (which did not make my EFI-based Ubuntu boot, although it booted normally when I did not use this manual selection). But maybe this only worked because my setup was (in principle) compatible with BIOS mode, with only the EFI-part in Ubuntu being in the way.
However, the real decision of whether you can get into BIOS mode or not is based on the structure of your MBR, which in turn depends on the partitioning tool you used. What you need is a so-called Hybrid MBR that combines two technologies in a hacky way (classical BIOS-style MBR and more modern GPT). I found that I always had this (after first preparing my Mac for dual boot, after wiping out my Mac OS, and also after re-partitioning the whole disk with the Ubuntu installer). However, if you think that you cannot get into BIOS mode because of this, then you should read about setting up a hybrid MBR with gdisk. You can already find out if you have a hybrid MBR without knowing much details by simply starting gdisk (installed on any live CD) and selecting your main device (e.g., /dev/sda); quit using "q".
Now I assume you got a (K)Ubuntu installer to boot in BIOS mode (verified by checking the absence of /sys/firmware/efi). Then you can just install the system as usual by following the installer. As usual, make a backup of your relevant data first. Fixing installation errors is much easier if you don't have to worry about wiping it all out again. By the way, I found that it is convenient to store a file with your manually installed packages for quicker re-install. You can get this by running (for example):
gunzip -c /var/log/apt/history.log.*.gz | grep "apt-get install" > my-past-install-commands.txt
The standard installation should go without errors. After that, your machine should boot successfully (and still in BIOS mode). Some more adjustments are in order:
Install NVidia drivers
Our main reason for all the hassle. Kubuntu will suggest these drivers to you by itself. You can also open "Driver manager" manually. Or just install the package "nvidia-331" (or similar with another version, or with "updates").
If Nouveau crashes your desktop so quickly that you do not manage to do this in time (it crashed seconds after login in my first setup), then you can either do the installation in recovery mode from the terminal, or switch off desktop effects. The latter made it a lot more stable for me (crashed only once every few hours). You can switch off the desktop effects in "System Settings" -> "Desktop Effects", or, if even this is impossible because of the crashing, then you can do it by editing the file ~/.kde/share/config/kwinrc in a text editor (possibly on the terminal) to change the line
under the first section [Compositioning] to
Switching off effects (composition) is also advisable when using live CDs/USBs to do recovery work, since they are also crashed by nouveau (but it really depends on how many applications/widgets you use).
When NVidia is installed correctly, you will see a brief NVidia splash before login, and you will be able to find "nvidia" as a module when running "lsmod|grep nvidia". With this driver, the GPU will not overheat all the time and suspend-to-ram will work out of the box.
Enabling brightness controls
Brightness controls (keyboard buttons at F1 and F2) will not work at first. To enable them, you have to create a file /usr/share/X11/xorg.conf.d/10-nvidia-brightness.conf with the following content:
Section "Device" Identifier "Device0" Driver "nvidia" VendorName "NVIDIA Corporation" BoardName "Quadro K1000M" Option "RegistryDwords" "EnableBrightnessControl=1" EndSection
You need to restart Xorg (or reboot) for this to take effect. Then brightness controls will just work.
Adjusting the touchpad settings
I found the touchpad to be far too sensitive (esp. for vertical scrolling). It seems there is a bug in the KDE touchpad settings that makes it impossible to enter sufficiently small values there. I solved this by creating a little script fixTouchpadSettings.sh in my home directory with the contents:
#!/bin/sh synclient VertScrollDelta=200 synclient HorizScrollDelta=200 synclient MaxTapMove=200 synclient MaxDoubleTapTime=100 synclient FingerLow=30 synclient FingerHigh=50
You can run this to see if the settings work for you as well, or you can run the commands individually. See the manual for synaptics. To make these adjustments "permanent", I registered this script as a KDE startup script in the System Settings. This seems to overwrite the KDE settings reliably (unless you change & apply the KDE settings after startup). Just don't forget that you did this in case you ever wonder why your touchpad does not keep its KDE settings after restart ;-).
Changing an existing EFI-based Ubuntu to use BIOS mode
Chances are that you came to this page because you already have a broken EFI-based installation and encountered some of the above errors. The question is: can you fix it now? You can try, but I have failed and found it faster to reinstall the system. You must make a backup of your data anyway at this point.
So here is what you could do:
- Check that you are really not in BIOS mode first (NVidia can fail for many reasons, and the errors are unspecific). See above for how to do this.
- Check if you have a hybrid MBR. See "gdisk" above for details.
- Then check out the instructions in this forum post (you might want to read the rest of that thread first for information on what you are about to try).
- If you already have hybrid MBR, your tasks reduces to uninstalling the EFI-grub and getting a BIOS-grub setup to work instead. I failed at that (installing grub on all kinds of partitions worked but it still would not boot).
- In Ubuntu, you need to uninstall (maybe with "--purge") packages "grub-efi-amd64" and "grub-efi-amd64-bin"; then install "grub-pc" instead.
- Maybe use "dpkg-reconfigure" to specify grub settings anew.
- When running grub manually from a changeroot environment, be sure to specify "--target=i386-pc" to clarify that you really don't want efi (even if your live disk was booted in EFI mode). Search the web for details.
I am deeply indebted to the thorough and knowledgeable people at Arch Linux who have had this discussion on how to fix the problem. It greatly increased my understanding of the issue and eventually allowed me to fix it. If I had more time now, I would seriously consider moving to Arch Linux ;-)
Comments and feedback for this article can be sent to me via email: markus at this domain.
You are giving a talk in 5 min. You attach your Kubuntu 14.04 LTS "Trusty Thar" laptop to the beamer. The laptop screen goes black for a second ... reappears ... and ... nothing. The beamer reports "no input". No message on your screen. 4min to go. Panic.
Keep cool. Do not reboot. Do not turn off the beamer. Both take too long and never fixed anything for me that was not fixed otherwise. Try the quick things first:With some luck, you can do this even if all screens are black. Still nothing? Read more …
Gender imbalance in academia is a huge topic these days, especially in the STEM fields. Computer Science, with its close connection to tech/Web/geek culture is buzzing with all kinds of related activities, funding programs, and (sometimes heated) debates. The statistics we get to see in these discussions are often very coarse, talking about all of Computer Science. Researchers thus tend to get the impression that their own (specific) field is merely suffering from wider issues beyond their control. So I was wondering how strong is gender imbalance in (really) specific research fields? Considering the conferences I go to, I can certainly say that some are more balanced than others. For journals, I have no idea. So I did a little data mining over the weekend to find out, using DBLP and Wikidata as main data sources.Read more …
When using BibTeX to create bibliographies with LaTeX, it is convenient to work with one large bib file that defines all relevant references, and that is re-used again and again in new LaTeX projects. Sometimes, however, such a large file is unwieldy, and you want to create a smaller file that contains only the entries that you have actually used in a document. I am hereby releasing a powerful software suite for solving this problem: BibDigest. Read more …
The Wikidata project – a major effort to establish a platform for editing and sharing data among all Wikipedia communities – will start next Monday, 2 April 2012, and we are already very busy drawing diagrams and discussing data semantics here in Berlin (I have the pleasure of being the Data Architect of the project, so I better get cracking to define some data model).
Today, Wikimedia Germany e.V. has also published there first official press release about the project, entitled Data Revolution for Wikipedia, and what an echo we get! The Techcrunch article is quite nice; the Twitter feedback on #wikidata is beyond my capacity. Now all that remains to be done is to actually implement it …
Technical details will be published on meta.wikimedia.org pretty much immediately. We will need strong support from the Semantic Web community to leverage this data. From the discussions we are having, it will be quite rich and more than your average RDF. Certainly some technical and academic challenges to be addressed there.
Next year's PODS 2012, the 31st edition of the established SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, is going to have a special call for "emerging database environments and applications" which happens to be very much geared to Semantic Web/Web of Data technologies. This would be a great chance to get some of the database/data mining/query-related work that is happening around semantic technologies to receive more visibility and appreciation in the database community.
The call for papers is found below. As you can see, more than one of the special areas could fit to (Semantic) Web-related work. Abstract submission is on November 20, so plenty of time to get ready :-) View call for papers …
While implementing RDF-based query answering for Semantic MediaWiki (SMW), I realized that the popular SPARQL query language for RDF has what I consider to be a major design flaw. Admittedly, I had been slightly too optimistic: surely, moving from SQL to SPARQL would be a piece of cake! Have we not had exactly the same RDF-ish data model in mind when designing SMW? And is SMW not a natural friend to all of its fellow Semantic Web technologies? Well, let's just say that the friendship has cooled down a bit in certain matters …Read more …
the past seven months with you as my single OS were an experience that I don't want to miss, but we just cannot go on like this. I'm exhausted by our quarrels about which window is in front. You wanted to lock me in, but I just felt locked out. I needed my freedom, you just gave me embedded X servers and virtual machines. Don't pretend to be surprised, I know why you wanted to prevent me from shrinking your partition! But your intrigues were in vain, and I have now returned to my first love, Kubuntu. Maybe we can see each other again from time to time. I still love your body, and Kubuntu is completely okay with this, as long as we don't multitouch. Farewell Snow Leopard! No more Cmd+Q disasters, no more "each tool has a different use for the home key" hassles, no more weekly updates for an iTunes I never use!
P.S. You still had all my emails, although you could never find them with your Spotlight. I moved them to my new home this morning while you were sleeping.
Aug 19 2010. I have recently moved from Linux to Mac, with all the advantages and disadvantages involved. In the process, I logged my customisation steps to create a Mac OS Installation Guide for Linux Users that can probably also be useful to other Linux users.
Aug 6 2010. I recently had to figure out how to migrate emails from KMail to Mac. Hopefully, my description can be of some use to others who move away from KMail for whatever reason. The basic method should work for moving KMail emails to virtually any other email client, and it can be used even if you cannot run KMail any more (as was the case for me since my Linux laptop passed away one morning without warning).
Older entries …