onyx.textdata.textfileprocess – >>> True

True

class onyx.textdata.textfileprocess.TFFinishedEvent

Bases: onyx.textdata.textfileprocess.TextFileEventBase

class onyx.textdata.textfileprocess.TFLineEvent

Bases: onyx.textdata.textfileprocess.TextFileEventBase

class onyx.textdata.textfileprocess.TFNewFileEvent

Bases: onyx.textdata.textfileprocess.TextFileEventBase

class onyx.textdata.textfileprocess.TextFileEventBase

Bases: object

class onyx.textdata.textfileprocess.TextFileProcessor(files, sendee=None, sending=True)

Bases: onyx.dataflow.streamprocess.ProcessorBase

From a list of text files, produce a stream of events corresponding to lines in the files. Separate events are sent for the beginning of each file and for reaching the end of the entire set (subsequent calls to process() continue to generate this event). Note that lines are returned with newlines still at the end, in the style of Python line-based file reading.

>>> module_dir, module_name = os.path.split(__file__)

>>> files = tuple(os.path.join(module_dir, text_file) for text_file in ('foo.txt', 'bar.txt', 'baz.txt'))
>>> result = []
>>> tfs = TextFileProcessor(files, sendee=result.append)
>>> while not tfs.finished():
...    tfs.process()
>>> result  
[TFNewFileEvent: ...foo.txt, TFLineEvent: [#!MLF!#
] (8), TFLineEvent: [foo 1
] (6), TFLineEvent: [foo 2
] (6), TFLineEvent: [foo 3
] (6), TFNewFileEvent: ...bar.txt, TFLineEvent: [bar 4
] (6), TFLineEvent: [bar 5
] (6), TFLineEvent: [bar 6
] (6), TFNewFileEvent: ...baz.txt, TFLineEvent: [baz 7
] (6), TFLineEvent: [baz 8
] (6), TFLineEvent: [baz 9
] (6), TFFinishedEvent]

>>> tfs.process()
>>> tfs.process()
>>> result[-3:]
[TFFinishedEvent, TFFinishedEvent, TFFinishedEvent]

dc

A debug context for this processor. This attribute is an object returned by dcheck() in the onyx.util.debugprint module, and may be used for debug output. The tag for turning on such output is available as debug_tag

debug_tag

Activate this tag to get debugging information, see onyx.util.debugprint.DebugPrint

finished()

graph

Return a graph for this processor. By default this is just a single node whose label is the label of processor; derived classes may wish to override this property.

label

Return a label for this processor. By default this is just the name of the class; derived classes may wish to override this property by providing a different label to __init__().

open_next_file()

process()

send(result)

Internal function that pushes result into the sendee. Implementations of process() must call this to push results. To set up the sendee, (the target of the push), clients of the processor must either initialize the object with a sendee, or call set_sendee(). Processors created with a sendee of False will never send, but will not error if send is called.

sendee

The callable this processor will use to send events; see set_sendee()

sending

Whether this processor will currently send events at all; see set_sending()

set_sendee(sendee)

Clients call this to set up the callable where the processor will send its results.

set_sending(sending)

Clients call this to turn sending from a processor on or off.

static std_process_prologue(process_function)

Subclasses may use this decorater on their process function to implement the usual bypass and process semantics and to set up the debug context returned by dc().