Constrained MDPs and the reward hypothesis

-
It's been a looong ago that I posted on this blog. But this should not mean the blog is dead. Slow and steady wins the race, right? Anyhow, I am back and today I want to write about constrained Markovian Decision Process (CMDPs). The post is prompted by a recent visit of Eugene Feinberg , a pioneer of CMDPs, of our department, and also by a growing interest in CMPDs in the RL community (see this , this , or this paper). For impatient readers, a CMDP is like an MDP except that there are multiple reward functions, one of which is used to set the optimization objective, while the others are used to restrict what policies can do. Now, it seems to me that more often than not the problems we want to solve are easiest to specify using multiple objectives (in fact, this is a borderline tautology!). An example, which given our current sad situation is hard to escape, is deciding what interventions a government should apply to limit the spread of a virus while maintaining economic
6 comments

I can run Matlab on my Mac again!

-
After much struggling today I managed to make Matlab run again my Mac.
The major problem was that Matlab complained about that I have the wrong version of X11 installed on my system and it won't start. As I have finished teaching today for the semester, I thought that I am going to celebrate this by resolving this issue which I was struggling with for a year or so by now. On the internet you will see a lot of advice on what to do, and as they say, the truth is indeed out there, however, it is not so easy to find. In a nutshell what seems to happen is this:

Why Matlab does not start when other applications do start (say, Gimp, Gnuplot using X11, etc.).
Matlab seems to make the assumption that the X11 libraries are located at /usr/X11/lib and it sticks to this assumption no matter how your system is configured. I use XQuartz and macports' X11 and they put stuff elsewhere. I had some legacy code sitting in /usr/X11/, which I did not use. It was a remainder of some version of X11 that I used probably 2 or 3 laptops ago. Matlab reported that the lib was found, but the "architecture was wrong". The error message had something like:

.. Did find: /usr/X11R6/lib/libXext.6.dylib: mach-o, but wrong architecture..

Anyhow, here is one solution.
You have to arrange that /usr/X11 points to a directory that has a working X11 copy.
It is probably a good idea to first clean up the old X11 installation. You can do this by following the advice on the XQuartz FAQ page by issuing the following commands in the terminal:
sudo rm -rf /usr/X11* /System/Library/Launch*/org.x.* /Applications/Utilities/X11.app /etc/*paths.d/X11 sudo pkgutil --forget com.apple.pkg.X11DocumentationLeo sudo pkgutil --forget com.apple.pkg.X11User sudo pkgutil --forget com.apple.pkg.X11SDKLeo sudo pkgutil --forget org.x.X11.pkg
Then I have reinstalled the latest XQuartz copy (not all these steps might be necessary, but in order to stay on the safe side, I will describe everything I did).
I also have macports and xorg-libX11, xorg-libXp, xorg-server seems necessary for the following steps to succeed (but possibly other xorg-* ports are also needed). I am guessing that XQuartz does not install all the libraries, but after installing enough xorg-* ports through macports, all the libraries will be installed which are used by Matlab.

Now, my X11 is located at /opt/X11 and some additional libs are found at /opt/local/lib.

So I created a bunch of symbolic links:

sudo ln -s /opt/X11 /usr/X11
for i in /opt/local/lib/libX* ; do sudo ln -s $i /usr/X11/lib; done

The first line creates a symbolic link to /opt/X11, while the second is necessary because of the additional libX* libraries which, for some reason, macports puts into /opt/local/lib instead of puttting it into /opt/X11/lib. Initially I did not know that I need these libs, and then Matlab complained that it did not find the image for some lib (it was /usr/X11/lib/libXp.6.dylib).

Anyhow, I am really happy that this worked!
I hope people who will have the same trouble will find my post useful.

Comments

  1. Tony5 March 2012 at 21:48

    Nice work! Thanks again for the help had this issue and your write up saved the day :D A+++

    ReplyDelete
  2. Anonymous30 October 2012 at 23:31

    I tried your approach without reinstalling XQuartz, but it didnt work. It seems that you posted exactly the order that it is needed: remove old XQuartz, reintall, install xorg-libX11, xorg-libXp, xorg-server, logout, open matlab x)

    ReplyDelete
  3. Unknown24 March 2014 at 04:25

    Hi,

    Thanks for explaining in details. I've tried your approach and it didn't work in my case. Do you have any suggestion?

    ReplyDelete
  4. Csaba Szepesvári24 March 2014 at 11:30

    Mansoureh Jesmani: I am sorry this did not work.
    Yasser al-hamad: Did you manage to solve your issues? What is exactly that you have problem with? (I do not have much time but I will try to help you if you drop me an email).

    ReplyDelete

Post a Comment

Popular posts from this blog

Constrained MDPs and the reward hypothesis

-
Image
It's been a looong ago that I posted on this blog. But this should not mean the blog is dead. Slow and steady wins the race, right? Anyhow, I am back and today I want to write about constrained Markovian Decision Process (CMDPs). The post is prompted by a recent visit of Eugene Feinberg , a pioneer of CMDPs, of our department, and also by a growing interest in CMPDs in the RL community (see this , this , or this paper). For impatient readers, a CMDP is like an MDP except that there are multiple reward functions, one of which is used to set the optimization objective, while the others are used to restrict what policies can do. Now, it seems to me that more often than not the problems we want to solve are easiest to specify using multiple objectives (in fact, this is a borderline tautology!). An example, which given our current sad situation is hard to escape, is deciding what interventions a government should apply to limit the spread of a virus while maintaining economic
Read more

Bayesian Statistics in Medical Clinical Trials

-
I came across a very interesting document. The document is titled " Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials ". It is a draft guidelines poster by the Center for Devices and Radiological Health of FDA, dated May 23, 2006. Why is this interesting? The job of FDA (the US Food and Drug Administration) is to make sure that the decisions in any clinical trial are made in a scientifically sound manner. Clearly, when following the Bayesian approach the choice of the prior and the model can influence the decisions. What does FDA do in this situation? The establish a process where they require a pre-specification (and agreement on) both the prior and the model, including an analysis of the operating characteristics of the design. This latter includes estimating the probability of erroneously approving an ineffective or unsafe device (the Type I error). This will typically be done by conducting Monte-Carlo simulations, where the Type I error is
Read more

Keynote vs. Powerpoint vs. Beamer

-
A few days ago I decided to give Keynote , Apple's presentation software, a try (part of iWork '09). Beforehand I used MS Powerpoint 2003, Impress from NeoOffice 3.0 (OpenOffice's native Mac version) and LaTeX with beamer . Here is a comparison of the ups and downs of these software, mainly to remind myself when I will reconsider my choice in half a year and also to help people decide what to use for their presentation. Comments, suggestions, critics are absolutely welcome, as usual. Btw, while preparing this note I have learned that go-oo.org has a native Mac Aqua version of OpenOffice. Maybe I will try it some day and update the post. It would also be good to include a recent version of Powerpoint in the comparison. Stability Keynote: Excellent After a few days of usage, so take this statement with a grain of salt.. MS Powerpoint 2003: Excellent Impress: Poor Save your work very often Beamer: Excellent Creating visually appealing slides, graphics on slides Keynote: Ex
Read more