NirspecPipeline

class msaexp.pipeline.NirspecPipeline(mode='jw02767005001-02-clear-prism-nrs1', files=None, verbose=True, source_ids=None, slitlet_ids=None, pad=0, positive_ids=False, primary_sources=True)

Bases: object

Container class for NIRSpec data, generally in groups split by grating/filter/detector

Parameters

modestr

Group / mode name, i.e., in the groups computed in msaexp.pipeline.exposure_groups

fileslist

Explicit list of exposure or slitlet files

verbosebool

Print status messages

source_idslist

Specific list of source_id to trim from the MSA metadata file

slitlet_ids: list

Specific list of slitlet_id to trim from the MSA metadata file

padint

Number of dummy slits to pad the open slitlets

positive_idsbool

If true, ignore background slits with source_id values <= 0

primary_sourcesbool

Only extract sources with primary_source='Y' in the MSA metadata file.

Attributes

modestr

Group name

fileslist

List of exposure (rate.fits) filenames

pipedict

Dictionary with data from the various calibration pipeline products. The final flux-calibrated data should generally be in pipe['phot'].

last_stepstr

The last step of the calibration pipeline that was run, e.g., ‘phot’

slitletsdict

Slitlet metadata

msametflstr

Filename of the MSAMETFL metadata file

msamsaexp.msa.MSAMetafile

MSA metadata object, perhaps that has been modified by the parameters source_ids, slitlet_ids and pad above

Attributes Summary

N	Number of exposures for this group
detector	Detector name, e.g., 'nrs1' from mode string
exp_type	Get data EXP_TYPE
grating	Grating name, e.g., 'prism' from mode string
is_fixed_slit	Are data in fixed-slit mode with EXP_TYPE == 'NRS_FIXEDSLIT'
targets	Reformatted target names for background and negative source names

Methods Summary

drizzle_2d(key[, drizzle_params])	Not used
extract_all_slits([keys, verbose, close])	Extract all spectra and make diagnostic figures
extract_spectrum(key[, slit_key, ...])	Main function for extracting 2D/1D spectra from individual slitlets
fit_profile(key[, yoffset, prof_sigma, ...])	Fit for profile width and offset
full_pipeline([load_saved, run_preprocess, ...])	Run all steps through extractions
get_background_slits(key[, step, ...])	Get background-subtracted slitlets
get_slit_polygons([include_yoffset])	Get slit polygon regions using slit wcs
get_slit_traces([verbose])	Set center of slit traces in slitlets.
initialize_from_cals([key, verbose])	Initialize processing object from cal.fits products
initialize_slit_metadata([use_yaml, ...])	Initialize the slitlets metadata dictionary
load_slit_data([step, verbose, indices, targets])	Load slitlet data from saved files.
parse_slit_info([write])	Parse information from / to {mode}.slits.yaml file.
preprocess([set_context, fix_rows, ...])	Run grizli exposure-level preprocessing
run_jwst_pipeline([verbose, run_flag_open, ...])	Steps taken from https://github.com/spacetelescope/jwebbinar_prep/blob/main/spec_mode/spec_mode_stage_2.ipynb
save_slit_data([step, verbose])	Save slit data to FITS
set_background_slits([find_by_id])	Initialize elements in self.pipe['bkg'] for background-subtracted slitlets
slit_index(key)	Index of key in self.slitlets.
slit_source_regions([color])	Make region file of source positions

Attributes Documentation

N

Number of exposures for this group

detector

Detector name, e.g., ‘nrs1’ from mode string

exp_type

Get data EXP_TYPE

Returns

exptstr

EXP_TYPE keyword from the first file in the file list. Returns an empty string '' if the file not found or of the keyword not found in the header.

grating

Grating name, e.g., ‘prism’ from mode string

is_fixed_slit

Are data in fixed-slit mode with EXP_TYPE == 'NRS_FIXEDSLIT'

targets

Reformatted target names for background and negative source names

• background_{i} to b{i}

• xxx_-{i} to xxx_m{i}

Methods Documentation

drizzle_2d(key, drizzle_params={}, **kwargs)

Not used

Drizzle the 2D spectra for a given slitlet.

Parameters

keystr

The key of the slitlet to drizzle the spectra for.

drizzle_paramsdict, optional

Additional parameters to pass to the ResampleSpecData class.

Returns

jwst.datamodels.ModelContainer

The drizzled 2D spectra.

extract_all_slits(keys=None, verbose=True, close=True, **kwargs)

Extract all spectra and make diagnostic figures

Parameters

keyslist, optional

List of keys corresponding to the slitlets to extract spectra from. If not provided, spectra will be extracted from all slitlets.

verbosebool, optional

Print status messages. Default is True.

closebool, optional

Close all figures after saving. Default is True.

**kwargsdict, optional

Additional keyword arguments to pass to the extract_spectrum method.

Returns

None

extract_spectrum(key, slit_key=None, prof_sigma=None, fit_profile_params={'min_delta': 100}, pstep=1.0, show_sn=True, flux_unit=Unit('uJy'), vmax=0.2, yoffset=None, skip=None, bad_dq_bits=1025, clip_sigma=-4, ntrim=5, get_slit_data=False, verbose=False, center2d=False, trace_sign=1, min_dyoffset=0.2, **kwargs)

Main function for extracting 2D/1D spectra from individual slitlets

Parameters

key: str

The key of the slitlet to extract the spectrum from.

slit_key: str

The key of the slit to use for extraction. If None, the last step in the pipeline will be used.

prof_sigma: float

The sigma parameter for the IntegratedGaussianPRF. If None, the value from the slitlet will be used.

fit_profile_params: dict

Additional parameters for fitting the profile. Default is {“min_delta”: 100}.

pstep: float

The step size for the running median. Default is 1.0.

show_sn: bool

Whether to show the signal-to-noise ratio. Default is True.

flux_unit: str

The unit of the flux. Default is FNU_UNIT.

vmax: float

The maximum value for the plot. Default is 0.2.

yoffset: float

The y offset for the slit. If None, the value from the slitlet will be used.

skip: list

A list of indices to skip. Default is None.

bad_dq_bits: int

The bad dq bits to mask. Default is (1 | 1024).

clip_sigma: float

The sigma value for clipping. Default is -4.

ntrim: int

The number of points to trim. Default is 5.

get_slit_data: bool

Whether to get the slit data. Default is False.

verbose: bool

Whether to print verbose output. Default is False.

center2d: bool

Whether to center the 2D spectrum. Default is False.

trace_sign: int

The sign of the trace. Default is 1.

min_dyoffset: float

The minimum value for the y offset. Default is 0.2.

Returns

If ‘key’ is not found in ‘slitlets’, returns None.

Else:

slitlet: object

The slitlet object.

tabs: list

A list of tables containing the extracted spectra.

full_tab: object

The combined table of all extracted spectra.

fig: object

The matplotlib figure object.

fit_profile(key, yoffset=None, prof_sigma=None, bounds=[(-5, 5), (0.5957446808510638, 1.4893617021276595)], min_delta=100, use_huber=True, verbose=True, **kwargs)

Fit for profile width and offset

Parameters

keystr

The key of the slitlet to fit the profile for.

yoffsetfloat, optional

The initial guess for the cross-dispersion offset. Default is None.

prof_sigmafloat, optional

The initial guess for the profile width. Default is None.

boundslist, optional

The bounds for the fitting parameters. Default is [(-5, 5), (1.4 / 2.35, 3.5 / 2.35)].

min_deltafloat, optional

The minimum change in chi-square required to perform the fit. Default is 100.

use_huberbool, optional

If True, use Huber loss function for fitting. Default is True.

verbosebool, optional

If True, print verbose output. Default is True.

Returns

tuple

A tuple containing the fitted profile width and offset.

full_pipeline(load_saved='phot', run_preprocess=True, run_extractions=True, indices=None, targets=None, initialize_bkg=True, make_regions=True, use_yaml_metadata=True, **kwargs)

Run all steps through extractions

Parameters

load_savedstr, optional

The calibration pipeline processing step to load saved data from. If provided, the pipeline will skip the preprocessing and JWST pipeline steps. Default is None.

run_preprocessbool, optional

Whether to run the preprocessing step. Default is True.

run_extractionsbool, optional

Whether to run the extraction step. Default is True.

indiceslist, optional

List of slit indices of specific files to process. Default is None.

targetslist, optional

List of target names of specific individual sources to process. Default is None.

initialize_bkgbool, optional

Whether to initialize the background slits. Default is True.

make_regionsbool, optional

Whether to create slit source regions. Default is True.

use_yaml_metadatabool, optional

Whether to use YAML metadata for initializing slit metadata. Default is True.

**kwargsdict, optional

Additional keyword arguments to pass to the preprocessing and extraction steps.

Returns

None

get_background_slits(key, step='bkg', check_background=True, **kwargs)

Get background-subtracted slitlets

Parameters

keystr

The key identifier of the slitlet

stepstr, optional

The step in the pipeline to get the slitlets from. Default is “bkg”

check_backgroundbool, optional

If True, check if the background subtraction has been performed. Default is True.

Returns

slitslist

List of jwst.datamodels.slit.SlitModel objects

get_slit_polygons(include_yoffset=False)

Get slit polygon regions using slit wcs

Deprecated, use regions_from_metafile.

get_slit_traces(verbose=True)

Set center of slit traces in slitlets.

Parameters

verbosebool, optional

If True, print verbose output. Default is True.

Returns

None

initialize_from_cals(key='phot', verbose=True)

Initialize processing object from cal.fits products

Parameters

keystr

The key to identify the calibration product to load. Default is “phot”.

verbosebool, optional

If True, print verbose output. Default is True.

Returns

None

initialize_slit_metadata(use_yaml=True, yoffset=0, prof_sigma=0.7659574468085106, skip=[])

Initialize the slitlets metadata dictionary

Parameters

use_yamlbool

Read data from stored yaml file

yoffsetfloat

Default cross-dispersion offset for source centering

prof_sigmafloat

Default profile width

skiplist

Exposures to skip

Returns

slitletsdict

Slitlet metadata

load_slit_data(step='phot', verbose=True, indices=None, targets=None)

Load slitlet data from saved files. This script runs msaexp.pipeline.SlitData for each exposure file in the group.

Parameters

stepstr

Calibration pipeline processing step

verbosebool

Print status messages

indiceslist, None

Optional slit indices of specific files

targetslist, None

Optional target names of specific individual sources

Returns

slit_listslist

List of msaexp.pipeline.SlitData objects containing the loaded slitlet data.

parse_slit_info(write=True)

Parse information from / to {mode}.slits.yaml file.

Parameters

writebool, optional

Whether to write the parsed information back to the YAML file. Default is True.

Returns

infodict

A dictionary containing the parsed information from the YAML file.

preprocess(set_context=True, fix_rows=False, scale_rnoise=True, skip_completed=True, **kwargs)

Run grizli exposure-level preprocessing

1. Snowball masking

2. Apply 1/f correction

3. Median “bias” removal

4. Rescale RNOISE

Parameters

set_contextbool

Set the CRDS_CTX based on the keyword in the exposure files

fix_rowsbool

Apply 1/f correction to detector rows, as well as columns

scale_rnoisebool

Calculate rescaling of the VAR_RNOISE data extension based on pixel statistics

skip_completedbool

Skip steps that have already been completed

Returns

statusbool

True if completed OK

run_jwst_pipeline(verbose=True, run_flag_open=True, run_bar_shadow=True, **kwargs)

Steps taken from https://github.com/spacetelescope/jwebbinar_prep/blob/main/spec_mode/spec_mode_stage_2.ipynb

See also https://jwst-pipeline.readthedocs.io/en/latest/jwst/pipeline/calwebb_spec2.html#calwebb-spec2

• AssignWcs: initialize WCS and populate slit bounding_box data

• Extract2dStep: identify slits and set slit WCS

• FlatFieldStep: slit-level flat field

• PathLossStep: NIRSpec path loss

• BarShadowStep: Bar shadow correction

• PhotomStep: Photometric calibration

Parameters

verbosebool

Printing status messages

run_flag_openbool

Run jwst.msaflagopen.MSAFlagOpenStep after AssignWcsStep

run_bar_shadowbool

Run jwst.barshadow.BarShadowStep after PathLossStep

Returns

statusbool

True if completed successfully

save_slit_data(step='phot', verbose=True)

Save slit data to FITS

Parameters

stepstr

The step of the pipeline from which the slit data is saved.

verbosebool, optional

If True, print verbose output. Default is True.

Returns

bool

True if the slit data is saved successfully.

set_background_slits(find_by_id=False)

Initialize elements in self.pipe['bkg'] for background-subtracted slitlets

Parameters

find_by_idbool, optional

If True, find background slits by source ID. If False, find background slits by slitlet ID. Default is False.

Returns

bool

True if the initialization is successful.

slit_index(key)

Index of key in self.slitlets.

Parameters

keystr

The key to search for in self.slitlets.

Returns

int or None

The index of key in self.slitlets if it exists, otherwise None.

slit_source_regions(color='magenta')

Make region file of source positions

Deprecated, see regions_from_metafile.