onyx.grid.gridmap – Object for doing gridmap work.

Support for transparently using the Sun Grid Engine (SGE) from Python code.

>>> map2 = GridMap(fake_grid=True, prefix='gridmap_y', working_dir=module_dir)
>>> map2.working_dir, map2.fake_grid 
('/.../py/onyx/grid', True)
>>> def foo(x): return len(x)
>>> res = map2.map_func(foo, tuple([None] * i for i in xrange(5))) 
Your job-array ....1-5:1 ("gridmap_y") has been submitted (qsubfake)
Job ... exited with exit code 0.
Job ... exited with exit code 0.
Job ... exited with exit code 0.
Job ... exited with exit code 0.
Job ... exited with exit code 0.
<BLANKLINE>
<BLANKLINE>
>>> res
[0, 1, 2, 3, 4]

>>> map = GridMap(fake_grid=True, working_dir=module_dir)
>>> map.working_dir, map.fake_grid 
('/.../py/onyx/grid', True)
>>> map._make_tasks((), ()) 
('/.../py/onyx/grid/gridmap...001.000', '/.../py/onyx/grid/gridmap...001.000/gridmap...001.000.job', ())

>>> map._make_tasks((), ()) 
('/.../py/onyx/grid/gridmap...001.001', '/.../py/onyx/grid/gridmap...001.001/gridmap...001.001.job', ())

class onyx.grid.gridmap.GridMap(working_dir='.', fake_grid=False, prefix='gridmap_x', min_process=3, max_task=10)

Bases: object

An object for doing vectorized map work on a compute grid.

fake_grid

static func3_spec(func)

grid_name

Return a name, based on self.prefix, that is suitable for use as a grid name. This means invalid characters are removed, the first character will be an alpha, and the length is limited.

>>> GridMap().grid_name
'gridmap_x'
>>> GridMap(prefix='').grid_name
'a'
>>> GridMap(prefix='0\x01/:@*?\\foo').grid_name
'a0foo'
>>> GridMap(prefix=' long rambling name ').grid_name
'longrambl'

instance

map_func(func, *arg_iters)

map_func4(func, *arg_iters)

map_func_t(func, args_iter)

map_func_t2(func, args_iter)

map_func_t3(func, args_iter)

map_func_t4(func, args_iter)

max_task

min_process

prefix

procid

qsub_cmd

working_dir