DIR is not the installation directory, it is the directory that contains the compiled source files (ie. /usr/src/mhash...).
You need to compile PHP with the --with-mhash[=DIR] parameter to enable this extension. DIR is the mhash installation directory.
As of PHP 5.3.0, the mhash extension is emulated thru the Hash extension. This makes the mhash installation directory have no effect, and requires the hash extension enabled to make the mhash support available.