python - Using multiprocessing in emcee library inside a class -
i have tried use emcee library implement monte carlo markov chain inside class , make multiprocessing module works after running such test code:
import numpy np import emcee import scipy.optimize op # choose "true" parameters. m_true = -0.9594 b_true = 4.294 f_true = 0.534 # generate synthetic data model. n = 50 x = np.sort(10*np.random.rand(n)) yerr = 0.1+0.5*np.random.rand(n) y = m_true*x+b_true y += np.abs(f_true*y) * np.random.randn(n) y += yerr * np.random.randn(n) class modelfit(): def __init__(self): self.x=x self.y=y self.yerr=yerr self.m=-0.6 self.b=2.0 self.f=0.9 def get_results(self): def func(a): model=a[0]*self.x+a[1] inv_sigma2 = 1.0/(self.yerr**2 + model**2*np.exp(2*a[2])) return 0.5*(np.sum((self.y-model)**2*inv_sigma2 + np.log(inv_sigma2))) result = op.minimize(func, [self.m, self.b, np.log(self.f)],options={'gtol': 1e-6, 'disp': true}) m_ml, b_ml, lnf_ml = result["x"] return result["x"] def lnprior(self,theta): m, b, lnf = theta if -5.0 < m < 0.5 , 0.0 < b < 10.0 , -10.0 < lnf < 1.0: return 0.0 return -np.inf def lnprob(self,theta): lp = self.lnprior(theta) likelihood=self.lnlike(theta) if not np.isfinite(lp): return -np.inf return lp + likelihood def lnlike(self,theta): m, b, lnf = theta model = m * self.x + b inv_sigma2 = 1.0/(self.yerr**2 + model**2*np.exp(2*lnf)) return -0.5*(np.sum((self.y-model)**2*inv_sigma2 - np.log(inv_sigma2))) def run_mcmc(self,nstep): ndim, nwalkers = 3, 100 pos = [self.get_results() + 1e-4*np.random.randn(ndim) in range(nwalkers)] self.sampler = emcee.ensemblesampler(nwalkers, ndim, self.lnprob,threads=10) self.sampler.run_mcmc(pos, nstep) test=modelfit() test.x=x test.y=y test.yerr=yerr test.get_results() test.run_mcmc(5000) i got error message :
file "mcmc_model.py", line 157, in run_mcmc self.sampler.run_mcmc(theta0, nstep) file "build/bdist.linux-x86_64/egg/emcee/sampler.py", line 157, in run_mcmc file "build/bdist.linux-x86_64/egg/emcee/ensemble.py", line 198, in sample file "build/bdist.linux-x86_64/egg/emcee/ensemble.py", line 382, in _get_lnprob file "build/bdist.linux-x86_64/egg/emcee/interruptible_pool.py", line 94, in map file "/vol/aibn84/data2/zahra/anaconda/lib/python2.7/multiprocessing/pool.py", line 558, in raise self._value cpickle.picklingerror: can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed i reckon has how have used multiprocessing in class not figure out how keep structure of class way , meanwhile use multiprocessing well??!!
i appreciate tips.
p.s. have mention code works if remove threads=10 last function.
there number of questions discuss what's going on:
…including one, seems response… same question:
however, difference here not using multiprocessing directly -- butemcee is. therefore, pathos.multiprocessing solution (from links above) not available you. since emcee uses cpickle, you'll have stick things pickle knows how serialize. out of luck class instances. typical workarounds either use copy_reg register type of object want serialize, or add __reduce__ method tell python how serialize it. can see several of answers above links suggest similar things… none enable keep class way have written it.
Comments
Post a Comment