Reference/API
Fitting
- mcfine.fitting.calculate_goodness_of_fit(data, error, best_fit_pars, n_comp)[source]
Calculate various goodness of fit metrics.
- mcfine.fitting.delta_bic_looper(data, error, fit_method='mcmc', initial_n_comp=None, initial_pars=None, save=False, overwrite=True, progress=False)[source]
Increase spectral complexity until we hit diminishing returns
This will iteratively build up the spectral model one component at a time, before looping backwards to remove the weaker components which might just be fits to the noise
- Parameters:
data – Observed data
error – Observed error
fit_method – “mcmc” or “leastsq”. “mcmc” will run emcee for each component and use fit parameters from that, whereas “leastsq” will only run an MCMC after a final fit has been found, to explore covariances. Defaults to “mcmc”.
initial_n_comp – Initial guess at number of components. Defaults to None.
initial_pars – Initial guess at parameters. Defaults to None.
save – Whether to save out or not, defaults to False
overwrite – Whether to overwrite existing fits. Defaults to False
progress – Whether to display progress bars or not. Defaults to False
- Returns:
fitted number of components, likelihood, and the emcee sampler
- mcfine.fitting.downsample(data, chunk_size=10, func=<function nanmean>)[source]
Downsample data, given a function
- Parameters:
data – Data to downsample. Should be an image
chunk_size – Chunk size to downsample to
func – Function to downsample using. Defaults to nanmean
- mcfine.fitting.emcee_wrapper(data, error, p0, n_walkers, n_dims, n_comp=1, progress=False)[source]
Light wrapper around emcee
The runs the emcee part, with some custom moves and passes the arguments neatly to functions
- Parameters:
data – Observed data
error – Observed uncertainty
p0 – Initial guess position for the walkers
n_walkers – Number of emcee walkers
n_dims – Number of dimensions for the problem
n_comp – Number of components to fit. Defaults to None
progress – Whether or not to display progress bar. Defaults to False
- mcfine.fitting.get_fits_from_samples(samples, vel, props, strength_lines, v_lines, fit_type='lte', n_draws=100, n_comp=1)[source]
Get a number of fit lines from an MCMC run
- Parameters:
samples – emcee output
vel – Velocity grid to evaluate the fit on
props – List of properties
strength_lines – List of line strengths
fit_type – Fit type. Defaults to “lte”
v_lines – List of line velocities
n_draws (int) – Number of draws to pull from samples
n_comp (int) – Number of components in the fit
- Returns:
array of best fit line
- mcfine.fitting.get_p0_lmfit(data, error, n_comp=1)[source]
Get initial guesses for parameters using lmfit
- Parameters:
data – Observed data
error – Observed uncertainty
n_comp – Number of components to fit. Defaults to 1
- mcfine.fitting.get_parameters_from_fitting(data, error, n_comp, p0_fit=None, fit_method='mcmc', save=False, overwrite=True, progress=False)[source]
Get fitted parameters, either using MCMC or least-squares
- Parameters:
data – Observed data
error – Observed error
n_comp – Number of components to fit
p0_fit – Initial guess of parameters for MCMC. If least-squares, will just use this. Defaults to None
fit_method – “mcmc” or “leastsq”. “mcmc” will run emcee for each component and use fit parameters from that, whereas “leastsq” will only run an MCMC after a final fit has been found, to explore covariances. Defaults to “mcmc”.
save – Whether to save out or not, defaults to False
overwrite – Whether to overwrite existing fits. Defaults to False
progress – Whether to display progress bars or not. Defaults to False
- Returns:
parameter medians
- mcfine.fitting.get_radex_multiple_components(theta, vel, v_lines, qn_ul)[source]
Wrapper around RADEX to get out the multiple components
- mcfine.fitting.initial_lmfit(params, intensity, intensity_err, vel, strength_lines, v_lines, props, n_comp=1, fit_type='lte', log_tau=True)[source]
Get initial guess for MC walkers from lmfit
This is the residual calculation for LMFIT. Just to get our initial parameters to instantiate the MC
- Parameters:
params – List of lmfit parameters
intensity – Measured intensity
intensity_err – Measured error on intensity
vel – Velocity axis
strength_lines – Relative line strength
v_lines – Relative line velocity
props – List of properties for the line profiles
n_comp – Number of components to fit. Defaults to 1
fit_type – Type of fit to do. Either ‘lte’ or ‘radex’. Defaults to ‘lte’
log_tau – Whether to fit tau in log-space (default) or linear space
- Returns:
Residual (chisq) value from the fit
- mcfine.fitting.initialise_positions(p0, n_walkers, n_dims, n_comp, data_percentage=False)[source]
Initialise positions for the walkers
- Parameters:
p0 – “best fit” values
n_walkers – number of walkers
n_dims – number of dimensions
n_comp – number of components
data_percentage – Whether to move walkers around based on a percentage of data. Defaults to False
- mcfine.fitting.ln_like(theta, intensity, intensity_err, vel, strength_lines, v_lines, props, n_comp=1, fit_type='lte', log_tau=True)[source]
Calculate the (negative) log-likelihood for the model
This is the main meat of the MCMC fitting, given the parameters we calculate the likelihood of the model and return that
- Parameters:
theta – Parameters for the fit
intensity – Observed intensity
intensity_err – Intensity uncertainty
vel – Observed velocity
strength_lines – Relative line strength
v_lines – Relative line velocities
props – List of properties being fit
n_comp – Number of components to fit. Defaults to 1
fit_type – Either ‘lte’ (default) or ‘radex’
log_tau – Whether to fit tau in log-space (default) or linear space
- Returns:
Negative ln-likelihood for the model.
- mcfine.fitting.ln_prob(theta, intensity, intensity_err, vel, strength_lines, v_lines, props, bounds, n_comp=1, fit_type='lte')[source]
Calculate the ln-probability for emcee
This combines the prior (generally flat) with the ln-likelihood to return to emcee
- Parameters:
theta – Parameters for the fit
intensity – Observed intensity
intensity_err – Intensity uncertainty
vel – Observed velocity
strength_lines – Relative line strength
v_lines – Relative line velocities
props – List of properties being fit
bounds – Bounds on parameters
n_comp – Number of components to fit. Defaults to 1
fit_type – Either ‘lte’ (default) or ‘radex’
- Returns:
Combined probability (likelihood+prior) for the model.
- Return type:
float
- mcfine.fitting.multiple_components(theta, vel, strength_lines, v_lines, props, n_comp, fit_type='lte', log_tau=True)[source]
Sum intensities for multiple lines.
Takes n_comp distinct lines, and calculates the total intensity of their various hyperfine lines.
- Parameters:
theta – [t_ex, tau, vel, vel_width] for each component. Should have a length of 4*`n_comp`
vel – velocity array
strength_lines – Array for relative line strengths
v_lines – Array of relative velocity shifts for the lines
props – List of properties for the line profiles
n_comp (int) – Number of distinct components to calculate intensities for
fit_type (str) – Should be one of ALLOWED_FIT_TYPES. Defaults to ‘lte’
log_tau (bool) – Whether to fit tau in log or linear space. Defaults to True (log space)
- Returns:
The sum of the intensities for all the distinct components.
- Return type:
np.ndarray
- mcfine.fitting.parallel_fit_samples(ij, fit_dict_filename=None, consolidate_fit_dict=True)[source]
Pull fit percentiles from a single pixel
Will pull from the fit dict if we can, else will read in each individual file
- mcfine.fitting.parallel_fitting(ij, fit_dict_filename='fit_dict', data_type='original', fit_method='mcmc', overwrite=False)[source]
Parallel function for MCMC fitting.
Wraps up the MCMC fitting to pass off to multiple cores. Because of overheads, it’s easier to farm out multiple fits to multiple cores, rather than run the MCMC with multiple threads.
- Parameters:
ij (tuple) – tuple containing (i, j) coordinates of the pixel to fit.
fit_dict_filename (str) – Base filename for MCMC ft pickle. Will append coordinates on afterward. Defaults to fit_dict.
data_type – Data type to fit. Can be either “original” or “downsampled”
fit_method – “mcmc” or “leastsq”. “mcmc” will run emcee for each component and use fit parameters from that, whereas “leastsq” will only run an MCMC after a final fit has been found, to explore covariances. Defaults to “mcmc”.
overwrite (bool) – Overwrite existing files? Defaults to False.
- Returns:
Number of fitted components and the best-fit likelihood.
- mcfine.fitting.radex_grid_interp(theta, qn_ul, labels=None)[source]
Interpolate generated RADEX grid to get useful values out without weird edge effects
- Parameters:
theta (list) – property values
qn_ul (list) – Names for the transitions
labels (list) – Labels for properties
- Returns:
tau and t_ex
- mcfine.fitting.run_mcmc(data, error, n_comp=1, save=True, overwrite=False, progress=False, fit_dict_filename='fit_dict.pkl', p0_fit=None)[source]
Run emcee to get a fit out
- Parameters:
data – Observed data
error – Observed uncertainty
n_comp – Number of components to fit. Defaults to 1
save – Whether to save the sampler out. Defaults to True
overwrite – Whether to overwrite existing fit. Defaults to False
progress – Whether to display progress bar. Defaults to False
fit_dict_filename – Name for the sampler. Defaults to fit_dict.pkl
p0_fit – Initial guess for the fit. Defaults to None, which will use some basic parameters
- mcfine.fitting.run_mcmc_sampler(data, error, p0, n_walkers, n_dims, n_comp=1, progress=False, pool=None)[source]
Tool for actually running emcee
There are two options here, fixed and adaptive which determine how long to run the MCMC walkers for. See the docs for more details
- Parameters:
data – Observed data
error – Observed uncertainty
p0 – Initial position guess for the walkers
n_walkers – Number of emcee walkers
n_dims – Number of dimensions for the problem
n_comp – Number of components to fit. Defaults to None
progress – Whether or not to display progress bar. Defaults to False
pool – If running in multiprocessing mode, this is the mp Pool
- mcfine.fitting.sample_tpeak_per_component(samples, n_comp, n_samples=500)[source]
Get a number of Tpeak samples per component
HyperfineFitter
- class mcfine.HyperfineFitter(data=None, vel=None, error=None, mask=None, config_file=None, local_file=None)[source]
- create_fit_cube(fit_dict_filename=None, mcfine_output_filename=None, cube_filename=None)[source]
Create upper/lower errors for spectral fit plots
Will preferentially get this from the mcfine_output, which is quicker and reduces I/O. Otherwise, will pull in each individual fit
If WCS is present, this will be saved out as a multi-extension fits. Otherwise, will save as a 4-D numpy array.
- Parameters:
fit_dict_filename (str) – Name for the filename of fitted parameter dictionary. Defaults to None, which will pull from config.toml
mcfine_output_filename (str) – Name for the mcfine output. Defaults to None, which will pull from config.toml
cube_filename (str) – Name for the filename of output cube. Defaults to None, which will pull from config.toml
- downsample_fitter(fit_dict_filename=None, mcfine_output_filename=None, downsample_factor=10)[source]
Run multicomponent fitter on downsampled data
This is a light wrapper around multicomponent fitter, but also takes
- Parameters:
fit_dict_filename – Filename to save the fitted emcee walkers to. Defaults to None, which will not save anything
mcfine_output_filename – Filename to save the mcfine output to. Defaults to None, which will not save anything
downsample_factor – Factor to downsample the data down on each spatial axis
- encourage_spatial_coherence(input_dir='fit', output_dir='fit_coherence', fit_dict_filename=None, mcfine_input_filename=None, mcfine_output_filename=None, reverse_direction=False)[source]
Loop over fits to encourage spatial coherence
This will loop over RA/Dec to potentially replace fits with neighbouring fits. This helps to encourage spatial coherence and can also help in the case of catastrophic misfits
- Parameters:
input_dir – directory for the input fits. Defaults to ‘fit’
output_dir – directory for the output of the coherence checking. Defaults to ‘fit_coherence’
fit_dict_filename – Filename structure for the fit dictionary. Default to None, which will choose some generic name
mcfine_input_filename – Name for the consolidated mcfine output to start with. Defaults to None, which will pull the default name “mcfine”
mcfine_output_filename – Name for the consolidated mcfine output to save at the end. Defaults to None, which will pull the default name “mcfine_coherence”
reverse_direction – Whether to reverse how we step through x/y in the coherence encouragement. Defaults to False, but likely you should run one case forward, then another backward
- generate_radex_grid(output_file=None)[source]
Pre-generate RADEX grid, given various bounds and steps
This will be the grid we interpolate over for speed later. As such, the parameters should be set to cover the parameter range you care about. Parameters are stored in config.toml
- Parameters:
output_file (str) – Output filename. Defaults to None, which will pull from config.toml
- get_props(fit_dict)[source]
Get properties out from a fit dictionary
- Parameters:
fit_dict – Dictionary of parameters
- initialise_mcfine_output(data=None, data_type='original')[source]
Initialise the mcfine output, which stores useful information about the fit run
- Parameters:
data – Array of data
data_type – Type of data. Should be “original” or “downsampled”. Will not take WCS for downsampled data
- make_parameter_maps(fit_dict_filename=None, mcfine_output_filename=None, maps_filename=None)[source]
Make maps of fitted parameters.
Will preferentially get this from the mcfine_output, which is quicker and reduces I/O. Otherwise, will pull in each individual fit
If WCS is defined, these will be .fits files, otherwise just numpy arrays
- Parameters:
fit_dict_filename (str) – Name for the filename of fitted parameter dictionary (without i/j suffix). Defaults to None, which will pull from config.toml
mcfine_output_filename (str) – Name for the mcfine output. Defaults to None, which will pull from config.toml
maps_filename (str) – Name for the filename of output maps. Defaults to None, which will pull from config.toml
- multicomponent_fitter(fit_dict_filename=None, mcfine_output_filename=None, data_type='original')[source]
Run the multicomponent fitter
This runs everything, essentially, and is the part that you should call. It’ll do LMFIT to get guesses, emcee to properly sample parameter space and then iteratively add components before removing them.
- Parameters:
fit_dict_filename – Filename to save the fitted emcee walkers to. Defaults to None, which will not save anything
mcfine_output_filename – Filename to save the mcfine output to. Defaults to None, which will not save anything
data_type – Data type to fit. Can be either “original” or “downsampled”
Plotting
- mcfine.plotting.parallel_corner(ij, plot_name='corner', fit_dict_filename=None, consolidate_fit_dict=True)[source]
Wrapper to parallelise corner plotting
- mcfine.plotting.parallel_plot_fit(ij, fit_dict_filename=None, consolidate_fit_dict=True, loc_image=None, n_comp=None, plot_name='fit', show_individual_components=True, show_hyperfine_components=False, n_points=1000, n_draws=100, figsize=(10, 4), x_label='Velocity (km s$^{-1}$)', y_label='Intensity (K)')[source]
Wrapper to parallelise fit spectrum plotting
- mcfine.plotting.parallel_step(ij, plot_name='step', fit_dict_filename='fit_dict', consolidate_fit_dict=True)[source]
Wrapper to parallelise step plotting
- mcfine.plotting.plot_fit(flat_samples, data, error, loc_image=None, i=None, j=None, n_comp=1, plot_name='fit', show_individual_components=True, show_hyperfine_components=False, n_points=1000, n_draws=100, figsize=(10, 4), x_label='Velocity (km s$^{-1}$)', y_label='Intensity (K)')[source]
Plot a fit spectrum
HyperfinePlotter
- class mcfine.HyperfinePlotter(data=None, error=None, mask=None, vel=None, config_file=None, local_file=None)[source]
- plot_corner(fit_dict_filename=None, mcfine_output_filename=None, plot_name=None, grid=None)[source]
Create corner plot for either a spectrum or a (subset of) a cube
- Parameters:
fit_dict_filename (str) – Name for the file containing the fit parameters. Defaults to None, which will pull from config.toml
mcfine_output_filename (str) – Name for the mcfine output. Defaults to None, which will pull from config.toml
plot_name (str) – Output plot name. Defaults to None, which will pull from config.toml
grid (np.ndarray) – If fitting a cube, can pass a ndarray of 1 (plot) and 0 (do not plot) to avoid making loads of plots. Defaults to None, which will then use the fitting mask and plot everything
- plot_fit(fit_dict_filename=None, mcfine_output_filename=None, plot_name=None, grid=None, loc_image=None)[source]
Create fit spectrum plot for either a single spectrum or a (subset of) a cube
- Parameters:
fit_dict_filename (str) – Name for the file containing the fit parameters. Defaults to None, which will pull from config.toml
mcfine_output_filename (str) – Name for the mcfine output. Defaults to None, which will pull from config.toml
plot_name (str) – Output plot name. Defaults to None, which will pull from config.toml
grid (np.ndarray) – If fitting a cube, can pass a ndarray of 1 (plot) and 0 (do not plot) to avoid making loads of plots. Defaults to None, which will then use the fitting mask and plot everything
loc_image (np.ndarray) – If fitting a cube, we can pass an image to here so the fit will also have a subplot showing the location of the pixel in question. Defaults to None, which will not plot this
- plot_step(fit_dict_filename=None, mcfine_output_filename=None, plot_name=None, grid=None)[source]
Create step plot for either a spectrum or a (subset of) a cube
- Parameters:
fit_dict_filename (str) – Name for the file containing the fit parameters. Defaults to None, which will pull from config.toml
mcfine_output_filename (str) – Name for the mcfine output. Defaults to None, which will pull from config.toml
plot_name (str) – Output plot name. Defaults to None, which will pull from config.toml
grid (np.ndarray) – If fitting a cube, can pass a ndarray of 1 (plot) and 0 (do not plot) to avoid making loads of plots. Defaults to None, which will then use the fitting mask and plot everything
emcee Functions
- mcfine.emcee_funcs.get_burn_in_thin(sampler, burn_in_frac, thin_frac)[source]
Get burn-in and thin from a sampler
- Parameters:
sampler – emcee sampler
burn_in_frac – burn-in fraction as a function of autocorrelation time
thin_frac – thin fraction as a function of autocorrelation time
- mcfine.emcee_funcs.get_samples(sampler, burn_in_frac, thin_frac)[source]
Get samples from an emcee sampler
- Parameters:
sampler – emcee sampler
burn_in_frac – burn-in fraction as a function of autocorrelation time
thin_frac – thin fraction as a function of autocorrelation time
- mcfine.emcee_funcs.get_samples_from_fit_dict(fit_dict, burn_in_frac, thin_frac)[source]
Pull out a bunch of samples from a fit dictionary
- Parameters:
fit_dict (dict) – Fit dictionary
burn_in_frac – burn-in fraction as a function of autocorrelation time
thin_frac – thin fraction as a function of autocorrelation time
fitting Functions
- mcfine.fitting_funcs.chi_square(observed, model, observed_error=None)[source]
Calculate standard chi-square.
If errors are provided, then this is the sum of \(({\rm obs}-{\rm model})^2/{\rm error}^2\). Else the sum of \(({\rm obs}-{\rm model})^2/{\rm model}^2\).
- Parameters:
observed (np.ndarray) – Observed values.
model (np.ndarray) – Model values.
observed_error (float or np.ndarray) – The error in the observed values. Defaults to None.
- Returns:
The chi-square value.
- Return type:
float
- mcfine.fitting_funcs.ln_prior(theta, vel, props, bounds, n_comp=1)[source]
Apply flat priors to the emcee fitting
This also ensures any velocity components are strictly increasing, to avoid degeneracies there
- Parameters:
theta – Parameters for the fit
vel – Observed velocity
props – List of properties being fit
bounds – Bounds on parameters
n_comp – Number of components to fit. Defaults to 1
- Returns:
0 if within bounds, -infinity otherwise
- mcfine.fitting_funcs.residual(observed, model, observed_error=None)[source]
Calculate standard residual.
If errors are provided, then this is the sum of \(({\rm obs}-{\rm model})/{\rm error}\). Else the sum of \(({\rm obs}-{\rm model})/{\rm model}\).
- Parameters:
observed (np.ndarray) – Observed values.
model (np.ndarray) – Model values.
observed_error (float or np.ndarray) – The error in the observed values. Defaults to None.
- Returns:
The residual value.
- Return type:
float
line shape Functions
- mcfine.line_shape_funcs.gaussian(x, amp, centre, width)[source]
Evaluate a Gaussian on a 1D grid.
Calculates a Gaussian, using \(f(x) = A \exp[-0.5(x - \mu)^2/\sigma^2]\).
- Parameters:
x (np.ndarray) – Grid to calculate Gaussian on.
amp (float or np.ndarray) – Height(s) of curve peak(s), \(A\).
centre (float or np.ndarray) – Peak centre(s), \(\mu\).
width (float or np.ndarray) – Standard deviation(s), \(\sigma\).
- Returns:
Gaussian model array
- Return type:
np.ndarray
- mcfine.line_shape_funcs.hyperfine_structure_lte(t_ex, tau, v_centre, line_width, strength_lines, v_lines, vel, return_hyperfine_components=False, log_tau=True)[source]
Create hyperfine intensity profile.
Takes line strengths and relative velocity centres, along with excitation temperature and optical depth to produce a hyperfine intensity profile.
- Parameters:
t_ex (float) – Excitation temperature (K).
tau (float) – Total optical depth of the line.
v_centre (float) – Central velocity of the strongest component (km/s).
strength_lines (np.ndarray) – Array of relative line strengths
v_lines (np.ndarray) – Array of relative velocity shifts for the lines (km/s)
vel (np.ndarray) – Velocity array (km/s)
line_width (float) – Width of components (assumed to be the same for each hyperfine component; km/s).
return_hyperfine_components (bool) – Return the intensity for each hyperfine component. Defaults to False.
log_tau (bool) – If True, will assume tau is in a logarithmic scale. Else is linear. Defaults to True.
- Returns:
- Intensity for each individual hyperfine component (if return_hyperfine_components is True), and the total
intensity for all components
- mcfine.line_shape_funcs.hyperfine_structure_pure_gauss(t, v_centre, line_width, strength_lines, v_lines, vel, return_hyperfine_components=False)[source]
Create hyperfine intensity profile for a pure Gaussian profile.
Takes line strengths and relative velocity centres, along with peak temperature to produce a hyperfine intensity profile.
- Parameters:
t (float) – Line temperature (K).
v_centre (float) – Central velocity of the strongest component (km/s).
strength_lines (np.ndarray) – Array of relative line strengths
v_lines (np.ndarray) – Array of relative velocity shifts for the lines (km/s)
vel (np.ndarray) – Velocity array (km/s)
line_width (float) – Width of components (assumed to be the same for each hyperfine component; km/s).
return_hyperfine_components (bool) – Return the intensity for each hyperfine component. Defaults to False.
- Returns:
- Intensity for each individual hyperfine component (if return_hyperfine_components is True), and the total
intensity for all components
- mcfine.line_shape_funcs.hyperfine_structure_radex(t_ex, tau, v_centre, line_width, v_lines, vel, return_hyperfine_components=False)[source]
Create hyperfine intensity profile using RADEX.
Takes line strengths and relative velocity centres calculated with RADEX, and produces a spectrum.
- Parameters:
t_ex (float) – Excitation temperature (K).
tau (float) – Total optical depth of the line.
v_centre (float) – Central velocity of the strongest component (km/s).
line_width (float) – Width of components (assumed to be the same for each hyperfine component; km/s).
v_lines (float) – Velocity of the various components (km/s).
vel (np.ndarray) – Velocity array (km/s)
return_hyperfine_components (bool) – Return the intensity for each hyperfine component. Defaults to False.
- Returns:
- Intensity for each individual hyperfine component (if return_hyperfine_components is True), and the total
intensity for all components
RADEX Functions
Utilities
- mcfine.utils.check_overwrite(val_dict, key)[source]
Check if overwrite is in a dict.
- Parameters:
val_dict (dict) – Dictionary of overwrites.
key (str) – Key to check.
- Returns:
Bool of whether to overwrite or not.
- mcfine.utils.get_dict_val(val_dict, default_val_dict, table, key, logger=None)[source]
Get value from a dictionary, falling to default if not present.
- Parameters:
val_dict (dict) – Provided dictionary of values
default_val_dict (dict) – Dictionary of fallback values
table (str) – Table key to search for
key (str) – Dictionary key to search for
logger – Optional logger instance
- Returns:
Dictionary value