hipercam

hipercam - a reduction package for the high-speed camera HiPERCAM

hipercam is a package for the reduction of data from the 5-band multi-window high-speed CCD camera HiPERCAM. It can access the raw data, display it, slice it into individual files containing the 5 CCDs, etc. It provides a combination of an API to access HiPERCAM data and high-level scripts to enact certain commonly-required operations in reduction. Most users will be most interested in the latter, and if that is the case for you, your first port of entry is probably to look at the ‘scripts’ sub-package, e.g. pydoc hipercam.scripts.

If you are interested in the API, then read on. The code is compartmentalised into several sub-modules and sub-packages, e.g. CCD and MCCD appear in hipercam.ccd, but they are also imported to the top level, so that e.g. pydoc hipercam.CCD and pydoc hipercam.ccd.CCD report back the same thing. This is true for most of the sub-modules, so looking through the top-level docs covers a large part of the API. However, it is rather long and if you want to specialise to a restricted part of the documentation, then a command like ‘pydoc hipercam.ccd’ can prove useful to find a more focussed view of related software. There are some exceptions though where you will need to drill deeper, namely the sub-packages like ‘ucam’ and ‘hcam’, and the plotting modules mpl and pgp.

Perhaps the most important class of all is MCCD which is the first place to go if you have a HiPERCAM ‘.hcm’ file you want to do something with beyond the current routines (although NB ‘.hcm’ files are just FITS files in thin disguise of a changed extension and you may be able to do your own stuff with standard tools for FITS files). Example code:

>> import hipercam as hcam
>> mccd = hcam.MCCD.read('myfile.hcm')
>> print(mccd)
>> print(mccd['1'])
>> print(mccd['1']['E1'])
>> print(mccd['1']['E1'][123,45])
>> mccd['1']['E1'][123,45] = 0
>> mccd.write('mynewfile.hcm')

which reads an hcm file in, then prints it, the CCD it contains labelled ‘1’ (a CCD object), then the window data from this CCD labelled ‘E1’ (a Window object) and finally the pixel value at iy=123, ix=45 (C-like starting from 0). This shows the dictionary-like interface used for MCCD and CCD objects alike (they are both sub-classes of a container class Group). It then sets this pixel to zero and writes out the result to a new file.

Submodules

Attributes

FIELD

HCAM

LIST

APER

HRAW

RED

LOG

DFCT

SEP

TBTS

DMINS

CNAMS

CIS

ALL_OK

NO_FWHM

NO_SKY

SKY_AT_EDGE

TARGET_AT_EDGE

TARGET_NONLINEAR

TARGET_SATURATED

REDUCE_FILE_VERSION

NO_EXTRACTION

NO_DATA

CLOUDS

JUNK

ANY_FLAG

BAD_FLAT

BAD_COLUMN

BAD_TIME

FLAGS

OUTLIER

FLAG_MESSAGES

FRNG

__version__

__all__

Exceptions

HipercamError

Class for the hipercam package errors

Classes

HipercamWarning

Class for hipercam package warnings. Use with warnings.warn

Group

A specialized OrderedDict that enforces string keys only and a

Agroup

A Group which defines arithmetic methods +=, +, etc which must be

Winhead

The header parts of a CCD window

Window

A CCD window, headers, position and data

CcdWin

All the Winhead objects of a single CCD.

MccdWin

All the Winhead objects for multiple CCDs.

CCD

Represents a single CCD as a Group of Window.

MCCD

Class representing a multi-CCD as a Group of CCD objects plus a FITS

Aperture

Represents an aperture for astronomical photometry

CcdAper

Class representing all the :class:Apertures for a single CCD.

MccdAper

Class representing all the :class:Apertures for multiple CCDs.

Target

Object to represent an astronomical target in terms of its appearance in a

Field

Object to represent a star field as a list of :class:`Target`s. This is to

Header

Simulates the basic functionality of astropy.io.fits.Header objects while

Functions

version()

Returns version number of installed HiPERCAM pipeline

gregorian_to_mjd(year, month, day)

This returns the Modified Julian Day number corresponding to the

mjd_to_gregorian(mjd)

This returns the year, month, and day corresponding to the supplied

fday_to_hms(fday)

Returns a tuple of (hours, minutes, seconds) given a fraction of a

get_ccd_info(fname)

Routine to return some useful basic information from an MCCD file without

trim_ultracam(mccd, ncol, nrow)

Trims columns and rows from an MCCD. The rows and columns are trimmed

Package Contents

hipercam.FIELD = '.fld'
hipercam.HCAM = '.hcm'
hipercam.LIST = '.lis'
hipercam.APER = '.ape'
hipercam.HRAW = '.fits'
hipercam.RED = '.red'
hipercam.LOG = '.log'
hipercam.DFCT = '.dft'
hipercam.SEP = '.sep'
hipercam.TBTS = '.tbts'
exception hipercam.HipercamError

Bases: Exception

Class for the hipercam package errors

class hipercam.HipercamWarning

Bases: astropy.utils.exceptions.AstropyUserWarning

Class for hipercam package warnings. Use with warnings.warn

hipercam.DMINS = 1440.0
hipercam.CNAMS
hipercam.CIS = ((1, 1, 1), (0, 0, 0), (0.898, 0, 0), (0.082, 0.69, 0.102), (0.012, 0.263, 0.875), (1, 0.008,...
hipercam.version()

Returns version number of installed HiPERCAM pipeline

hipercam.gregorian_to_mjd(year, month, day)

This returns the Modified Julian Day number corresponding to the start of the Gregorian date supplied. Algorithm taken from Duffet-Smith. Tested against astropy.time.Time. Designed to be simple and fast.

Parameters:
yearint

Year (valid range: 1582 onwards)

monthint

Month of year, 1 to 12 inclusive.

dayint

day of month (1 to 31; will not check month by month)

Returns:
mjdint

Modified Julian Day number

hipercam.mjd_to_gregorian(mjd)

This returns the year, month, and day corresponding to the supplied MJD. From Duffet-Smith. Checked against ‘gregorian_to_mjd’.

Parameters:
mjdint

Modified Julian Day number

Returns:
Returns a tuple of (year,month,day)
hipercam.fday_to_hms(fday)

Returns a tuple of (hours, minutes, seconds) given a fraction of a day. The hours and minutes are integers, the seconds are floating point.

fday should lie in the interval [0,1)

hipercam.ALL_OK
hipercam.NO_FWHM
hipercam.NO_SKY
hipercam.SKY_AT_EDGE
hipercam.TARGET_AT_EDGE
hipercam.TARGET_NONLINEAR
hipercam.TARGET_SATURATED
hipercam.REDUCE_FILE_VERSION = '20230222'
hipercam.NO_EXTRACTION
hipercam.NO_DATA
hipercam.CLOUDS
hipercam.JUNK
hipercam.ANY_FLAG
hipercam.BAD_FLAT
hipercam.BAD_COLUMN
hipercam.BAD_TIME
hipercam.FLAGS
hipercam.OUTLIER
hipercam.FLAG_MESSAGES
hipercam.FRNG = '.frng'
class hipercam.Group(ftype, *args, **kwargs)

Bases: collections.OrderedDict

A specialized OrderedDict that enforces string keys only and a single class for the values. The objects stored as values should support a copy method to return a deepcopy for use in the Group copy operation.

Group is designed to amalgamate several of the classes used for objects associated with CCDs such as Window (see CCD), and Aperture (see CcdAper)

ftype
__setitem__(key, item)

Adds an item item keyed by key checking that its type matches

copy(memo=None)

Copy operation. The stored objects must have a copy(self, memo) method.

check()

Checks validity of a Group if its objects have a method ‘clash’ defined to check that they are compatible. This should have signature ‘clash(self, other)’, but does not have to be defined for all objects storable in Groups. See Winhead for an example of an object for which this makes sense.

If the Group fails this check, a ValueError is raised.

get_num(num)

Returns the num element of the Group (starting from 0)

__copy__()

Copy operation for copy.copy

__deepcopy__(memo)

Copy operation for copy.deepcopy

__repr__()

Return repr(self).

class hipercam.Agroup(ftype, *args, **kwargs)

Bases: Group

A Group which defines arithmetic methods +=, +, etc which must be supported by whatever objects the Group contains. This allows the same approach to be taken when operating on CCD and MCCD objects which are based on this class. The objects stored in the Agroup must define a method copy which returns a deep copy; this is required by several operators.

copy(memo=None)

Copy operation.

__iadd__(other)

Adds other to the Agroup as ‘self += other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to add to each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__isub__(other)

Subtracts other from the Agroup as ‘self -= other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to subtract from each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__imul__(other)

Multiplies the Agroup by other as ‘self *= other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to multiply each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__itruediv__(other)

Divides the Agroup by other as ‘self /= other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to divide into each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__add__(other)

Adds other to the Agroup as ‘= self + other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to add to each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__sub__(other)

Subtracts other from the Agroup as ‘= self - other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to subtract from each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__mul__(other)

Multiplies the Agroup by other as ‘= self * other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to multiply each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__truediv__(other)

Divides the Agroup by other as ‘= self / other’. If other is another Agroup with the same object type (ftype) as self, then the operation will be applied to each pair of objects with matching keys. Otherwise other will be regarded as a constant object to divide into each object in the Agroup.

In the first case, if self has keys that are not in other, then they will be left untouched. Any keys in other not in self are ignored.

__radd__(other)

Adds the Agroup to other as ‘= other + self’.

__rsub__(other)

Subtracts the Agroup from other as ‘= other - self’.

__rmul__(other)

Multiplies the Agroup by other as ‘= other * self’.

__rtruediv__(other)

Divides a Agroup by other as ‘= other / self’.

__repr__()

Return repr(self).

class hipercam.Winhead(llx, lly, nx, ny, xbin, ybin, outamp, head=None, copy=False)

Bases: Header

The header parts of a CCD window

Winhead objects contain everything needed to represent a CCD window other than its data. This represents an arbitrary rectangular region of binned pixels. The lower-left pixel of the CCD is assumed to have coordinates (x,y) = (1,1). Winhead dimensions are in binned pixels.

>>> from hipercam import Winhead
>>> win = Winhead(12, 6, 100, 150, 2, 3)
>>> print(win)
Parameters:
llxint

X position of lower-left pixel of window (unbinned pixels)

llyint

Y position of lower-left pixel of window (unbinned pixels)

nxint

X dimension of window, binned pixels

nyint

Y dimension of window, binned pixels

xbinint

Binning factor in X

ybinint

Binning factor in Y

outampstr, {‘’,’LL’,’LR’,’UL’,’UR’}

Location of output amplifier. Options: ‘’ == unknown; ‘LL’ == lower-left; ‘LR’ == lower-right; ‘UL’ == upper-left; ‘UR’ == upper-right. Used when trimming to remove the correct part of the window.

headHeader, optional

Arbitrary header items (excluding ones such as LLX reserved for containing the above parameters when reading and writing Winhead objects). This will be deep copied to avoid different Winheads sharing the same header. See ‘copy’ for how the header is tranferred into the Winhead

copybool, optional

Controls whether the header is copied by value (True) or simply by reference (False). The latter is lightweight and faster but can cause unexpected behaviour especially when constructing multiple Winhead (or Window) objects if one repeatedly sends the same Header to each of them.

Attributes:
llxint

X ordinate lower-left pixel, unbinned pixels

llyint

Y ordinate lower-left pixel, unbinned pixels

xbinint

X-binning factor

ybinint

Y-binning factor

nxint

X-dimension, binned pixels

nyint

Y-dimension, binned pixels

outampstr, {‘’,’LL’,’LR’,’UL’,’UR’}

output amplifier location

urxint

X-ordinate of unbinned pixed at upper right of Winhead (starts from 1 on the left)

uryint

Y-ordinate of unbinned pixed at upper right of Winhead (starts from 1 on the left)

xlofloat

Left-most edge of window. Will be something-point-five as the centre of pixels are integers.

xhifloat

Right-most edge of window.

ylofloat

Lower edge of window

yhifloat

Upper edge of window

llx
lly
xbin
ybin
outamp
_nx
_ny
property nx

Binned X-dimension of the Winhead.

property ny

Binned Y-dimension of the Winhead.

__repr__()
__str__()
format(nohead=False)

Used to ensure that only the Winhead format gets printed which is useful in some instances. Relying on __repr__ carries the risk of being overloaded. Set ‘nohead’ True to suppress the header in addition

property urx

Unbinned X pixel at upper-right of Winhead

property ury

Unbinned Y pixel at upper-right of Winhead

property xlo

Left-hand edge of window (llx-0.5)

property xhi

Right-hand edge of window (urx+0.5)

property ylo

Bottom edge of window (lly-0.5)

property yhi

Top edge of window (ury+0.5)

x(xpix)

Given an X-pixel position, returns the physical X in the CCD.

Arguments:

xpix : (float / ndarray)
  X-pixel position in terms of binned pixels. Centre of left-most
  pixel is 0.

Returns the physical location measured in unbinned pixels, with the centre of left-most pixels of the CCD = 1.0

y(ypix)

Given a Y-pixel position, returns the physical Y in the CCD.

Arguments:

ypix : (float / ndarray)
  Y-pixel position in terms of binned pixels. Centre of lowest
  pixel is 0.

Returns the physical location measured in unbinned pixels, with the centre of lowest pixels of the CCD = 1.0

x_pixel(x)

The inverse of x: returns the X-pixel position given a physical X location.

Arguments:

x : (float)
  the physical location measured in unbinned pixels, with the centre
  of left-most pixels of the CCD = 1.0

Returns the X-pixel position in terms of binned pixels. Centre of the left-most pixel is 0.

y_pixel(y)

The inverse of y: returns the Y-pixel position given a physical Y location.

Arguments:

Y : (float)
  the physical location measured in unbinned pixels, with the centre
  of lowest pixels of the CCD = 1.0

Returns the Y-pixel position in terms of binned pixels. Centre of the lowest pixel is 0.

extent()

Returns (left,right,bottom,top) boundaries of Winhead i.e. (xlo,xhi,ylo,yhi)

outside(win)

Returns True if self contains the :class:Winhead win in such a way that it could be cut down to it. This implies that even if binned, its pixels are “in step”, aka “synchronised”, and that its binning factors are integer divisors of those of win.

Arguments:

win  : :class:Winhead
   the :class:Winhead that we are testing against to see if `self`
   surrounds it.

See also inside(), window()

inside(win)

Returns True if win contains self in such a way that it could be cut down to it. This implies that even if binned, its pixels are “in step”, aka “synchronised”, and that its bin factors are integer divisors of those of self.

Arguments:

win  : :class:Winhead
   the :class:Winhead that we are testing against to see if `self` in
   inside it.

See also outside(), window()

xy()

Returns two 2D arrays containing the x and y values at the centre of each pixel defined by the Winhead. See numpy.meshgrid to see what this means.

clash(win)

Raises a ValueError if two :class: Winhead`s are considered to ‘clash’. In this case this means if they have any pixels in common. This method is used in the ‘check’ method of the :class:`Group class to check the mutual validity of a set of Winhead.

Arguments:

win : :class:`Winhead`
   the :class:`Winhead` that we are testing self against.
matches(win)

Tests that the Winhead matches another. If all OK, returns None, otherwise raises a ValueError reporting the two Winhead`s. See also `__eq__

Arguments:

win : :class:`Winhead`
   the :class:`Winhead` that we are testing self against.
copy(memo=None)

Returns a copy (deepcopy) of the Winhead

copy.copy and copy.deepcopy of a Winhead use this method

distance(x, y)

Calculates the minimum distance of a point from the edge of the Winhead. If the point is outside the Winhead the distance will be negative; if inside it will be positive. The edge is defined as the line running around the outside of the outer set of pixels. For a point outside the box in both x and y, the value returned is a lower limit to the distance.

window(xlo, xhi, ylo, yhi, copy=False)

Generates a new Winhead by windowing it to match the complete pixels visible within the range xlo to xhi, ylo to yhi.

Arguments:

xlo : float
   minimum X, unbinned pixels (extreme left pixels of CCD centred
   on 1). None to ignore.

xhi : float
   maximum X, unbinned pixels. None to ignore.

ylo : float
   minimum Y, unbinned pixels (bottom pixels of CCD centred on 1)
   None to ignore.

yhi : float
   maximum Y, unbinned pixels. None to ignore

copy : bool
   controls whether the header is copied over, or just
   referenced.  The latter is more efficient, but if you later want
   to change the header could propogate those changes to whatever
   you copied.

Returns the windowed Winhead. Raises a ValueError if there are no visible pixels.

__copy__()
__deepcopy__(memo)
__eq__(win)

Defines equality. Two :class:`Winhead`s are equal if they match exactly (same lower left corner, dimensions and binning factors)

Arguments:

win : :class:`Winhead`
   the :class:Winhead that we are testing self against.
__ne__(win)

Defines equality. Two :class:`Winhead`s are equal if they match exactly (same lower left corner, dimensions and binning factors)

Arguments:

win : :class:`Winhead`
   the :class:Winhead that we are testing self against.
class hipercam.Window(win, data=None, copy=False)

Bases: Winhead

A CCD window, headers, position and data

Constructed from a Winhead and a numpy.ndarray which is stored in an attribute called data.

>>> import numpy as np
>>> from hipercam import Winhead, Window
>>> win = Winhead(12, 6, 100, 150, 2, 3, 'LL')
>>> data = np.ones((150,100))
>>> wind = Window(win,data)
>>> wind += 0.5
>>> wind *= 2

You cannot directly change the nx, ny values of a Window; you have to change its data array attribute and nx and ny will be taken from it.

Window objects support various arithematical operations such as subtraction or addition of constants. The end result of these always has a float type for safety to avoid problems with e.g. trying to make the result of adding a float to an integer an integer or with the range of integers.

Parameters:
winWinhead

the Winhead defining the position and, optionally, headers

data2D numpy.ndarray

the data (2D). The dimensions must match those in win unless data is None in which case a zero array of the correct size will be created. A ValueError will be raised if not. Standard C-type ordering, y-then-x assumed.

copybool

flag sent to Winhead controlling whether the header is copied by value or reference. ‘False’ is lightweight and faster but when building up CCDs from multiple Windows, you should probably use ‘True’ unless you are careful to make ‘win’ a different object every time.

property nx

Returns binned X-dimension of the Window.

property ny

Returns binned Y-dimension of the Window.

flatten()

Return data of the Window as a 1D array

classmethod rhdu(hdu)

Constructs a Window from an ImageHdu. Requires header parameters ‘LLX’, ‘LLY’, ‘XBIN’ and ‘YBIN’ to be defined. This converts the data to float32 internally, unless it is read in as float64 in the first place conversion which would lose precision.

whdu(head=None, xoff=0, yoff=0, extnam=None)

Writes the Window to an astropy.io.fits.ImageHDU

Arguments:

head : astropy.io.fits.Header
    Extra header items to add at the start of header in addition to
    those already contained in the :class:`Window`.

xoff : int
    Offset in X-direction to use for mosaicing.

yoff : int
    Offset in Y-direction to use for mosaicing.

extnam : None | string
    Extension name, useful in 'fv'

Returns:

hdu    : astropy.io.fits.ImageHDU
   The HDU containing the data and the header.
property size

Number of pixels

property winhead

A copy of the Winhead underlying the Window This is to allow simpler reporting of problems with the format without the data as well

set_const(val)

Sets the data array to a constant

add_noise(readout, gain)

Adds noise to a Window according to a variance calculated from V = readout**2 + counts/gain. Arguments:

readout : (float)
    RMS readout noise in counts

gain : (float)
    Gain in electrons per count.
min()

Returns the minimum value of the Window.

max()

Returns the maximum value of the Window.

mean()

Returns the mean value of the Window.

median()

Returns the median value of the Window.

sum()

Returns the sum of the Window.

std()

Returns the standard deviation of the Window.

percentile(q)

Computes percentile(s) of a Window.

Arguments:

q : float or sequence of floats
  Percentile(s) to use, in range [0,100]
add_fxy(funcs, ndiv=0)

Routine to add in the results of evaluating a function or a list of functions of x & y to the Window. Each function must take 2D arrays of x and y values for each pixel of the Window and return an array of values for each point. If you have lots of things to add, this routine saves some overhead by only generating the x,y pixel arrays once at the start. Pixels can be subdivided ndiv per unbinned pixel (ndiv == 0 simply computes the result at pixel centre)

Arguments:

funcs : (a callable or a list of callables)
    the callable(s) must have signature::

       arr = func(x,y)

    where x and y are 2D arrays containing the x and y values at the
    centre of each pixel in the :class:`Window`. Each func must have
    a method 'offset(self, dx, dy)' which returns a copy of the
    function displaced in centre by dx, dy unbinned pixels.

ndiv : (int)

    a sub-division factor used to improve the photometric accuracy
    when pixellation matters. The funcs are computed over a grid of
    ndiv*ndiv points per unbinned pixel.
copy(memo=None)

Returns a copy (deepcopy) of the Window

copy.copy and copy.deepcopy of a Window use this method

window(xlo, xhi, ylo, yhi, copy=False)

Creates a new Window by windowing it down to whatever complete pixels are visible in the region xlo to xhi, ylo to yhi. ValueError raised if no Window left.

Arguments:

xlo : float
   minimum X, unbinned pixels (extreme left pixels of CCD
   centred on 1)

xhi : float
   maximum X, unbinned pixels

ylo : float
   minimum Y, unbinned pixels (bottom pixels of CCD centred on 1)

yhi : float
   maximum Y, unbinned pixels

copy : bool
   controls whether the headers in the Winhead associated
   with this are copied by value or reference. True=copy by
   value which can carry significant overheads, but has the
   virtue of creating an independent object.

Returns the windowed Window.

crop(win)

Creates a new :class:Window by cropping the current :class:Window to the format defined by :class:Winhead win. Will throw a ValueError if the operation is impossible or results in no overlap. The current :class:Window must lie outside win and must be synchronised (in step) if any binning is used. If binning is used then the binning factors of the :class:Window must be divisors of the binning factors of those of win. If binning takes place it will be carried out by averaging, as appropriate for cropping flat-field frames (but not star fields).

Arguments:

win : (:class:Winhead)
   the new format to apply.
float32()

Converts the data type of the array to float32.

float64()

Converts the data type of the array to float64.

uint16()

Converts the data type of the array to uint16. A warning will be issued if there will be loss of precision. A ValueError is thrown if any data are outside the range 0 to 65535 This is to save space on output.

search(fwhm, x0, y0, thresh, fft, max=False, percent=50.0)

Search for a target in a :class:Window. Works by convolving the image with a gaussian of FWHM = fwhm, and returns the location of the maximum in the smoothed image which exceeds a level thresh and lies closest to the expected position. The convolution improves the reliability of the identification of the object position and reduces the chance of problems being caused by cosmic rays, although if there is more overall flux in a cosmic ray than the star, it could go wrong.

This routine is intended to provide a first cut in position for more precise methods to polish.

Arguments:

 fwhm : float
   Gaussian FWHM in pixels. If <= 0, there will be no convolution,
   although this is not advisable as a useful strategy.

 x0 : float
   x-position to judge position from (CCD-coordinates). The closest
   sufficiently high maximum will be taken.

 y0 : float
   y-position to judge position from (CCD-coordinates). The closest
   sufficiently high maximum will be taken.

 thresh : float
   The peak counts above background in the maximum of the *smoothed*
   image must exceed this value for a maximum to count. Use this to
   filter out noise.

 fft : bool
   The astropy.convolution routines are used. By default FFT-based
   convolution is applied as it scales better with fwhm, especially
   for fwhm >> 1, however the direct method (fft=False) may be faster
   for small fwhm values and images and has a better behaviour at the
   edges where it extends value with the nearest pixel while the FFT
   wraps values.

max : bool
   If True, just go for the highest peak, i.e. ignore x0, y0. The peak
   should still exceed the background by `thresh`

percent : float
   percentile to use to compute the background value. < 0 and it will
   be set to the minimum. 50% = median by default.

Returns:

a tuple of (x,y,peak): x,y is the location of the
brightest pixel measured in terms of CCD coordinates
(i.e. lower-left pixel is at (1,1)) and `peak` is the image value
at the peak pixel, in the *unconvolved* image. It might be useful
for initial estimates of peak height. If no peak is found, a
HipercamError will be raised.
__copy__()
__deepcopy__(memo)
__repr__()
__iadd__(other)

Adds other to the Window as ‘wind += other’. other can be another Window, or any object that can be added to a numpy.ndarray.

__isub__(other)

Subtracts other from the Window as ‘wind -= other`. other can be another Window, or any object that can be subtracted from a numpy.ndarray.

__imul__(other)

Multiplies the Window by other as ‘wind *= other’. other can be another Window, or any object that can multiply a numpy.ndarray.

__itruediv__(other)

Divides other into the Window as ‘wind /= other`. other can be another Window, or any object that can be divided into a numpy.ndarray.

__add__(other)

Adds other to a Window as wind + other. Here other can be a compatible Window (identical window) or any object that can be added to a numpy.ndarray, e.g. a float, another matching array, etc.

__radd__(other)

Adds other to a Window as other + wind. Here other is any object that can be added to a numpy.ndarray, e.g. a float, another matching array, etc.

__sub__(other)

Subtracts other from a Window as wind - other. Here other can be a compatible Window (identical window) or any object that can be subtracted from a numpy.ndarray, e.g. a float, another matching array, etc.

__rsub__(other)

Subtracts a Window from other as other - wind. Here other is any object that can have a numpy.ndarray subtracted from it, e.g. a float, another matching array, etc.

__mul__(other)

Multiplies a Window by other as wind * other. Here other can be a compatible Window (identical window) or any object that can multiply a numpy.ndarray, e.g. a float, another matching array, etc.

__rmul__(other)

Multiplies a Window by other as other * wind. Here other is any object that can multiply a numpy.ndarray, e.g. a float, another matching array, etc.

__truediv__(other)

Divides a Window by other as wind / other. Here other can be a compatible Window (identical window) or any object that can divide into a numpy.ndarray, e.g. a float, another matching array, etc.

For safety, the data array of the result will have a float type.

__rtruediv__(other)

Divides other by a Window as other / wind. Here other is any object that can be divided by a numpy.ndarray, e.g. a float, another matching array, etc.

class hipercam.CcdWin(wins=Group(Winhead))

Bases: Group

All the Winhead objects of a single CCD.

Acts as a container for all of the window formats of a CCD.

Parameters:
winsGroup(Winhead)

All of the Winhead objects

__repr__()
toJson(fname)

Dumps ccdWin in JSON format to a file

Arguments:

fname : (string)
   name of file to dump to
classmethod fromJson(fname)

Read from JSON-format file fname

Returns a CcdWin object.

class hipercam.MccdWin(wins=Group(CcdWin))

Bases: Group

All the Winhead objects for multiple CCDs.

Parameters:
winsGroup(CcdWin)

All of the CcdWin objects

__repr__()
toJson(fname)

Dumps MccdWin in JSON format to a file

Arguments:

fname : (string)
   file to dump to
classmethod fromJson(fname)

Read from JSON-format file fname

Returns an MccdWin object.

classmethod fromMccd(mccd)
class hipercam.CCD(winds, nxtot, nytot, nxpad=0, nypad=0, copy=False)

Bases: hipercam.group.Agroup

Represents a single CCD as a Group of Window.

CCD objects support a few operations such as returning the mean value, arithematic operations, and file I/O to/from FITS files. Header information is passed via the Windows. The first of these is assumed to be the location for any data pertaining to the CCD as a whole.

nxtot
nytot
nxpad = 0
nypad = 0
__reduce__()

This is to overcome a problem with pickling CCDs. The arguments for init don’t get picked up for some reason so this must return them in the second element of the tuple

flatten()

Returns all data of a CCD as a single 1D array, analogous to numpy.flatten

min()

Returns the minimum value of the CCD, i.e. the minimum of all the minimum values of all the Window

max()

Returns the maximum value of the CCD.

mean()

Returns the mean value of the CCD.

median()

Returns the median value of the CCD.

percentile(q, xlo=None, xhi=None, ylo=None, yhi=None)

Computes percentile(s) of the CCD.

Arguments:

q : float or sequence of floats
  Percentile(s) to use, in range [0,100]

xlo : int | None
  To restrict range of pixels

xhi : int | None
  To restrict range of pixels

ylo : int | None
  To restrict range of pixels

yhi : int | None
  To restrict range of pixels
whdul(hdul=None, cnam=None, xoff=0, yoff=0)

Write the CCD as a series of HDUs, one per Window, adding to and returning an HDUList.

Arguments:

hdul : astropy.io.fits.HDUList
   The HDUList to add to.

cnam : string | None
   CCD name.

xoff : int [if cname is not None]
   X-offset for mosaicing in ds9. Ignored if cnam is None.

Returns:: an HDUList.

classmethod rhdul(hdul, cnam=None)

Builds a single CCD from an HDUList.

If cnam is None, it is assumed that each HDU represents a Window. Each data HDU will be searched for a header parameter ‘WINDOW’ to label the Window, but the routine will attempt to generate a sequential label if WINDOW is not found. If the auto-generated label conflicts with one already found, then a KeyError will be raised.

If cnam is not None, cnam is taken to be the label of a CCD to be extracted from the HDUList which may contain multiple CCDs. It is assumed in this case that the CCD of interest has a continguous set of HDUs all containing the keyword CCD set equal to cnam. This allows individual CCDs to be read from hcm MCCD files.

Arguments:

hdul  : :class:`HDUList`
     each HDU will be read as sub-window of the :class:`CCD`

cnam  : (None | string)
     if cnam is not None, the routine will try to find and return a
     :class:CCD built from a contiguous subset of HDUs that matches
     cnam (the keyword CCD is used)

Returns a CCD. Data are converted to float32 unless they are read in as float64.

classmethod rmhdul(hdul)

Builds a CCD from an HDUList potentially containing multiple CCDs, i.e. an MCCD object. This function acts as a generator to allow the CCDs to be returned in successive calls. It is assumed that hdul is positioned at the start of the CCDs (i.e. with the initial data-less primary HDU removed).

Arguments:

hdul  : :class:`HDUList`
     each ImageHDU will be read as a sub-window of the :class:`CCD`

Returns:

A series of (label, :class:`CCD`) 2-element tuples to be used as
as follows::

  >> for label, ccd in CCD.rmhdul(hdul, True):
  >>    ... do something

Data are converted to float32 unless they are read in as float64.

write(fname, overwrite=False)

Writes out the CCD to a FITS file.

Arguments:

fname : (string)
     Name of file to write to.

overwrite : (bool)
     True to overwrite pre-existing files
classmethod read(fname, cnam=None)

Builds a CCD from a FITS file. Expects a primary HDU, containing no data, followed by a series of HDUs each containing data for a series of non-overlapping windows.

This can also be applied to extract a particular CCD, labelled ‘cnam’ from an MCCD hcm file, on the assumption that all HDUs for a given CCD come in a contiguous block with each one labelled with the keyword ‘CCD’.

Data are converted to float32 unless they are read in as float64.

matches(ccd)

Check that the CCD matches another, which in this case means checking that each window of the same label matches the equivalent in the other CCD.

There will be no match if there any windows present in one CCD but not the other and the windows must have identical location, size and binning.

Raises KeyError or ValueError exception depending on the problem.

copy(memo=None)

Make a copy of the CCD

copy.copy and copy.deepcopy of a CCD use this method

float32()

Applies :class:Window.float32 to all Windows of a CCD

float64()

Applies :class:Window.float64 to all Windows of a CCD

uint16()

Applies :class:Window.uint16 to all Windows of a CCD

set_const(val)

Sets all Windows to a constant value

inside(x, y, dmin)

Tests whether a point x,y lies within any Window of a CCD, at least dmin from its outer edge. It returns with the Window label or None if the point is not inside any Window

crop(ccd)

Given a template CCD called ccd, this tries to modify the format of the CCD, returning a new CCD. This is often needed e.g. prior to applying a full frame flat field. In general this operation requires re-labelling of the Window composing the CCD. It raises a HipercamError if it fails.

property head

Returns the first Window or an empty fits.Header if there is no first Window. The header of the first Window is where general header items of the CCD are stored.

is_data()

Returns True / False according to whether the frame is thought to contain data. Uses DSTATUS keyword if present, else returns True

__repr__()

Return repr(self).

class hipercam.MCCD(ccds, head=None, copy=False)

Bases: hipercam.group.Agroup

Class representing a multi-CCD as a Group of CCD objects plus a FITS header. It supports arithematic operations and file I/O to/from FITS files.

head = None
__reduce__()

This is to overcome a problem with pickling MCCDs. The arguments for init don’t get picked up for some reason so this must return them in the second element of the tuple

write(fname, overwrite=False, xgap=200, ygap=200)

Writes out the MCCD to a FITS file.

Arguments:

fname : string or file-like object
    Name of file to write to. Can also be a file, opened in
    writeable binary mode.

overwrite : bool
    True to overwrite pre-existing files

xgap  : int
   X-gap used to space CCDs for ds9 mosaicing (unbinned pixels)

ygap  : int
   Y-gap used to space CCDs for ds9 mosaicing (unbinned pixels)
classmethod read(fname)

Builds an MCCD from a FITS file. Expects a primary HDU, containing general headers and the first HDU, followed by a series of HDU blocks for each CCD with the first one of each containing some general per-CCD header items.

Data are converted to float32 unless they are read in as float64.

classmethod rhdul(hdul)

Builds an MCCD from an HDUList.

This will usually be passed the HDUList from a file. The header from the first (primary) HDU will be used to create the header for the MCCD. It is then assumed that the data for each CCD is contained in blocks of HDUs after the first. The data from all HDUs will be read into the Window objects that make up the CCD. Each HDU will be searched for a header parameter WINDOW to label the Window, but the code will attempt to generate a sequential label if WINDOW is not found.

Arguments:

hdul : :class:`HDUList`
     each ImageHDU will be read as sub-window of the :class:`CCD`

Data are converted to float32 unless they are read in as float64.

copy(memo=None)

Make a copy of the MCCD

copy.copy and copy.deepcopy of an MCCD use this method

matches(mccd)

Check that the MCCD matches another, which in this means checking that each CCD of the same label matches the equivalent in the other MCCD.

There will be no match if there any CCDs present in one MCCD which are not in the other.

Raises KeyError or ValueError exception depending on the problem.

crop(mccd)

Crops the :class:MCCD to the same format as a template, mccd. Returns the new version. Will raise a HipercamError if it fails.

__repr__()

Return repr(self).

float32()

Applies :class:Window.float32 to all Windows of an MCCD

float64()

Applies :class:Window.float64 to all Windows of an MCCD

uint16()

Applies :class:Window.uint16 to all Windows of an MCCD

set_const(val)

Sets all CCDs to a constant value

hipercam.get_ccd_info(fname)

Routine to return some useful basic information from an MCCD file without reading the whole thing in. It returns an OrderedDict keyed on the CCD label which returns the maximum X and Y dimensions of the CCD and X and Y padding as a 4-element tuple. If no CCD label [header parameter ‘CCD’] is found, the routine assigns a label of ‘1’ on the assumption that the file cotains a single CCD.

hipercam.trim_ultracam(mccd, ncol, nrow)

Trims columns and rows from an MCCD. The rows and columns are trimmed from each nearest to the readout amplifier as indicated by the value of the “outamp” attribute. e.g. if outamp=’LL’ then the rows and columns are stripped from the bottom and the left of the window respectively. This is useful because for ULTRACAM and ULTRASPEC these rows and columns can suffer from ringing effects.

Arguments:

mccd : MCCD
  the MCCD to be trimmed. Modified in place, i.e. it's changed on
  output.

ncol : int
  number of columns to be trimmed.

nrow : int
  number of rows to be trimmed
class hipercam.Aperture(x, y, rtarg, rsky1, rsky2, ref, compo=False, mask=[], extra=[], link='')

Represents an aperture for astronomical photometry

Essentially this consists of 3 circles representing the object, radius rtarg, and the sky annulus between radii r2 and r3, centered at a specific position, but there are several extra characteristics in addition. These are:

Reference apertures: these indicate (typically) bright stars that should be easy to locate. The idea is that when re-positioning apertures, you might want to initially do the brightest stars, and then see what the overall x,y shift is before attempting fainter ones. This status is indicated with a logical flag.

Linked apertures: some targets are nearly impossible to register, or may fade so much that they cannot be detected. Such targets can be “linked” to others in the sense that they are offset from them.

COMPO apertures: comparison stars injected onto the science image by COMPO have a difference world coordinate system to the main science image. Their motion on the CCD is flipped, so that an (x, y) shift in the stars on the science CCD produces a (-x, -y) shift in the COMPO stars. This status is indicated with a logical flag.

Sky masks: (‘mask) these are fixed circles offset from the aperture in question indicating pixels to ignore when estimating the sky background.

Star masks: (‘extra’) these are circles offset from aperture indicating pixels to include when summing the object flux. This is to combat problems caused by blended objects. These also act as sky masks so there should be no need to also mask such object.

Parameters:
xfloat

X position of centre of aperture, or the X offset to apply if the aperture is linked from another.

yfloat

Y position of centre of aperture, or the Y offset to apply if the aperture is linked from another.

rtargfloat

Radius (unbinned pixels) of target aperture

rsky1float

Inner radius (unbinned pixels) of sky annulus

rsky2float

Outer radius (unbinned pixels) of sky annulus

refbool

True/False to indicate whether this is a reference aperture meaning that its position will be re-determined before non-reference apertures to provide a shift.

compobool

True/False to indicate whether this is a compo aperture meaning that its shifts are inverted w.r.t non-compo apertures.

masklist of 3 element tuples

Each tuple in the list consists of an x,y offset and a radius in unbinned pixels. These are used to mask nearby areas when determining the sky value (e.g. to exclude stars)

extralist of 2 element tuples

Similar to mask, but each tuple in the list consists only of an x,y offset. These however are used as centres of additional target apertures to allow blended stars to be include in the total flux. They are given the same radius as the target aperture. They are also used to exclude sky pixels.

linkstr

If != ‘’, this is a string label for another Aperture that this Aperture is linked from. The idea is that this label can be used to lookup the Aperture.

Note

Normal practice would be to set link, mask, extra later, having created the Aperture. Attributes of the same name as all the arguments are defined. We copy the mask and extra apertures to avoid propagating references.

x
y
rtarg
rsky1
rsky2
ref
compo = False
mask
extra
__repr__()
copy(memo=None)

Returns with a copy of the Aperture

add_mask(xoff, yoff, radius)

Adds a mask to the :class:Aperture

add_extra(xoff, yoff)

Adds a mask to the :class:Aperture

Links this :class:Aperture to a lookup label for another

Cancels any link to another :class:Aperture

property linked

Returns True if the :class:Aperture is linked to another

check()

Run a few checks on an :class:Aperture. Raises a ValueError if there are problems.

write(fname)

Dumps Aperture in JSON format to a file called fname

toString()

Returns Aperture as a JSON-type string

classmethod read(fname)

Read from JSON-format file fname

class hipercam.CcdAper(aps=Group(Aperture))

Bases: Group

Class representing all the :class:Apertures for a single CCD. Normal usage is to create an empty one and then add apertures via the usual mechanism for updating dictionaries, i.e. ccdap[label] = aperture.

__repr__()
check()

Checks for problems with links

write(fname)

Dumps ccdAper in JSON format to a file called fname

copy(memo=None)
class hipercam.MccdAper(aps=Group(CcdAper))

Bases: Group

Class representing all the :class:Apertures for multiple CCDs. Normal usage is to create an empty one and then add apertures via the usual mechanism for updating dictionaries, e.g.

>> mccdap = MccdAper() >> mccdap[‘ccd1’] = CcdAper() >> mccdap[‘ccd2’] = CcdAper() >> mccdap[‘ccd1’][‘ap1’] = Aperture(100,200,10,15,125,False)

etc.

__repr__()
write(fname)

Dumps MccdAper in JSON format to a file called fname

toString()

Returns MccdAper in JSON format as a string

classmethod read(fname)

Read from JSON-format file fname. Since such files can fairly easily be corrupted by injudicious editing, some consistency checks are run. File loading does not happen often, so this should not be a serious overhead

fp : a file-like object opened for reading of text

Returns an MccdAper object.

class hipercam.Target(xcen, ycen, height, fwhm1, fwhm2, angle, beta, fmin)

Object to represent an astronomical target in terms of its appearance in a CCD image. The representation chosen is 2D “Moffat” function, which takes the form h/(1+(r/r0)**2)**beta, where ‘h’ is the central height, ‘r’ is the distance from the centre, ‘r0’ is a scale, and ‘beta’ is an exponent which for large values makes the profile Gaussian.

To allow for trailing, poor focus and to roughly simulate galaxies, Target objects can be elliptical in shape.

xcen
ycen
height
fmin
_fwhm1
_fwhm2
_angle
_beta
copy(memo=None)

Returns a copy of the Target

__copy__()
__deepcopy__(memo)
_comp_abc()
property fwhm1
property fwhm2
property angle
property beta
offset(dx, dy)

This generates a new Target by offsetting the position of the Target

Arguments:

dx : float
  X offset to add to the positions of all targets [unbinned pixels]

dy : float
  Y offset to add to the positions of all targets [unbinned pixels]

Returns:: a shifted version of the Target

add(wind, scale=1.0)

Adds the Target to a Window with an optional scaling factor. Returns 1 if anything added, 0 if there was no overlap.

Arguments:

wind : Window
  a Window, i.e. an array that knows where it is located.

scale : float
  factor to scale how much is added in.
__repr__()
class hipercam.Field

Bases: list

Object to represent a star field as a list of :class:`Target`s. This is to help with simulation of star fields.

add_random(ntarg, x1, x2, y1, y2, h1, h2, angle1, angle2, fwhm1, fwhm2, beta, fmin)

Adds a random field of ntarg Target objects scattered over the range x1, x2, y1, y2, with peak heights from h1 to h2, random angles from angle1 to angle2, but with fixed width parameters. The peak heights are chosen between the limits to have a distribution appropriate to a 3D Euclidean distribution of constant luminosity objects. The distributions over x, y and angle are uniform.

Arguments:

ntarg : (int)
   number of targets to add

x1 : (float)
   left limit of region to add targets to

x2 : (float)
   right limit of region to add targets to

y1 : (float)
   lower limit of region to add targets to

y2 : (float)
   upper limit of region to add targets to

h1 : (float)
   lowest peak height

h2 : (float)
   highest peak height

angle1 : (float)
   lowest angle degrees clockwise from the X axis

angle2 : (float)
   highest angle degrees clockwise from the X axis

fwhm1 : (float)
   FWHM along major axis of objects

fwhm2 : (float)
   FWHM along minor axis of objects

beta : (float)
   exponent that appears in Moffat functions

fmin : (float)

   parameter used to restrict region affected by a target: it's the
   minimum count level to which the profile will be computed.
modify(transform, fscale)

This generates a new Field by applying transform to the x,y locations of each target to change them. It also scales the targets’ peak heights by a constant factor fscale.

Arguments:

transform : (callable)
    Should have signature transform(x,y) and when given the centre
    x,y of a target, it should return the change in x,y needed to
    get to the new position which will be fed to Target.offset

fscale : (float)
    Scaling factor  It should also have an attribute `fscale`
    that is a scaling factor that will be applied to the targt peak
    heights.

See hipercam.command_line.makedata for an example.

add(wind, ndiv=0)

Adds all the targets of a Field to a Window

Arguments:

wind : Window
   targets will be added to it; modified on exit

ndiv : int
   sub-division factor to account for pixellation
wjson(fname)

Writes a Field to a file in json format. This is provided as a straightforward way to store all the info of a Field.

Arguments:

fname : (string)
   file to write to
classmethod rjson(fname)

Creates a Field from a json format file as saved by wjson.

Arguments:

fname : (string)
   the file to read to create the :class:`Field`
class hipercam.Header(head=None, copy=False)

Simulates the basic functionality of astropy.io.fits.Header objects while being lighter weight and thus more efficient. The idea is to allow headers that are compatible with writing to and reading from FITS files, i.e. that have (uppercase) keywords, values, comments and allows for ‘HIERARCH’ long keywords. History and comments are added at the end of the header. There is no attempt to replicate all the methods of astropy.io.fits.Header, although many are similar in nature to reduce the number of changes to the code required to implement this. This class has does not have all the checks of astropy.io.fits.Header objects so it is possible to create invalid headers, but this should become obvious when attempting conversion to FITS. Some example code:

>> import Header from hipercam
>> head = Header()
>> head['a'] = (1.23, 'a comment')
>> head['object'] = 'IP Peg'

Header objects are used as the base class of Winhead and Window objects which build into CCD and MCCD objects.

SPECIAL_KEYWORDS = ('COMMENT', 'HISTORY', '')
property to_fits

Returns the Header as an astropy.io.fits.Header which is useful when saving to disk

static _process_key(key)

This converts keys of 8 characters or less to uppercase. Any leading ‘HIERARCH ‘ is stripped

__setitem__(key, item)

Sets the value of a header item using the [] form, e.g. head[‘NREC’] = 2345. Reserved keywords ‘COMMENT’, ‘HISTORY’ and ‘’ (blank) are trapped.

Arguments:

key : string
   the name of the header item.

item : tuple | string | int | float
   the (value,comment) pair or just the value (in which
   case the comment will be blank.
__getitem__(key)

Returns the value associated with header item ‘key’ using the [] form, e.g. nrec = head[‘NRECORD’].

get(key, default)

Returns the value associated with header item ‘key’, returning a default value if the header item does not exist.

get_full(key)

Returns with 2-element (value,comment) tuple for the given keyword

get_comment(key)

Returns with the comment for the given keyword

add_comment(comment)

Adds a comment to the end of the ‘COMMENT’ section

add_history(history)

Adds a line of history to the end of the ‘HISTORY’ section

copy()
__repr__()
__delitem__(key)

Deletes a particular header item when called e.g. as del head[key].

__contains__(key)

Defines the operation ‘in’ for a Header. Look up whether a given key is in the header

update(head)

Update the Header with the contents of another header (excluding comments and history)

hipercam.__version__
hipercam.__all__