HTK SCP files are just lists of filenames, one per line. The ScpProcessor takes these lines, opens and reads the file, and generates an HTKAudioBlockEvent for each file.
>>> scpProc = ScpProcessor(prefix_subst=('/Users/kbasye1/work/R1/mfcc//ind_trn/adg0_4',
... onyx.home + '/data/htk_r1_adg0_4'))
BlockToFeatures is a simple processor which takes HTKAudioBlockEvents and generates feature vectors as Numpy arrays.
>>> result = []
>>> b2f = BlockToFeatures(sendee=result.append)
>>> scpProc.set_sendee(b2f.process)
>>> module_dir, module_name = os.path.split(__file__)
>>> scp_file_name = os.path.join(module_dir, "ind_trn109.scp")
>>> with open(scp_file_name) as scp_file:
... limited_iter = islice(scp_file, 10) # only process the first 10 lines
... source = IteratorSource(limited_iter, scpProc.process)
... source.start()
... source.wait_iter_stopped()
... source.stop(flush=True)
... source.wait()
... source.done()
True
>>> len(result)
3294
>>> print result[0]
[ -1.57193413e+01 -5.69683504e+00 -1.42094164e+01 -1.09782257e+01
-9.74443531e+00 -6.75594759e+00 -2.05153537e+00 -2.27887034e+00
-8.51778793e+00 -1.37430739e+00 7.37931538e+00 4.42070436e+00
-1.05129251e+01 -3.37096214e-01 8.85067165e-01 1.51379669e+00
8.33324611e-01 -4.58665192e-01 8.61221790e-01 5.57038009e-01
5.02852559e-01 -6.86278343e-01 -1.18701637e+00 -3.33604860e+00
-4.29879606e-01 0.00000000e+00 -1.03502944e-01 -9.11712088e-03
2.38455529e-03 3.19893450e-01 2.66819388e-01 6.52765259e-02
1.64075077e-01 3.10689837e-01 7.19753325e-01 4.09678102e-01
5.58297873e-01 3.91171537e-02 0.00000000e+00]
>>> data = load_scp_file(scp_file_name, max_lines=20,
... prefix_subst=('/Users/kbasye1/work/R1/mfcc//ind_trn/adg0_4',
... onyx.home + '/data/htk_r1_adg0_4'))
>>> print len(data)
20
>>> fname0 = data[0][0]
>>> os.path.basename(fname0)
'adg0_4_sr009.mfc'
>>> os.path.basename(data[-1][0])
'adg0_4_st0358.mfc'
Regression test for max_lines=0
>>> data = load_scp_file(scp_file_name, max_lines=0,
... prefix_subst=('/Users/kbasye1/work/R1/mfcc//ind_trn/adg0_4',
... onyx.home + '/data/htk_r1_adg0_4'))
>>> data
[]
Bases: onyx.dataflow.streamprocess.ProcessorBase
Transform HTKAudioBlockEvents into a stream of raw features.
Events sent: Numpy arrays of features
Transform HTKAudioBlockEvents into a stream of raw features.
sendee is a function to call with our output events.
A debug context for this processor. This attribute is an object returned by dcheck() in the onyx.util.debugprint module, and may be used for debug output. The tag for turning on such output is available as debug_tag
Activate this tag to get debugging information, see onyx.util.debugprint.DebugPrint
Return a graph for this processor. By default this is just a single node whose label is the label of processor; derived classes may wish to override this property.
Return a label for this processor. By default this is just the name of the class; derived classes may wish to override this property by providing a different label to __init__().
Process an HTKAudioBlockEvent, generate feature vectors and push them on.
Internal function that pushes result into the sendee. Implementations of process() must call this to push results. To set up the sendee, (the target of the push), clients of the processor must either initialize the object with a sendee, or call set_sendee(). Processors created with a sendee of False will never send, but will not error if send is called.
The callable this processor will use to send events; see set_sendee()
Whether this processor will currently send events at all; see set_sending()
Clients call this to set up the callable where the processor will send its results.
Clients call this to turn sending from a processor on or off.
Bases: object
Event corresponding to a block of audio from an HTK source.
Generated, e.g., by reading through an scp file, see ScpProcessor.
Bases: onyx.dataflow.streamprocess.ProcessorBase
Generator for blocks of HTK audio data from a stream of scp lines.
Event types sent: HTKAudioBlockEvent
sendee is a function to call with our output events. prefix_subst, if provided, is a tuple of two strings. Any lines in the file which have the first string as a prefix will have that prefix replaced by the second string.
A debug context for this processor. This attribute is an object returned by dcheck() in the onyx.util.debugprint module, and may be used for debug output. The tag for turning on such output is available as debug_tag
Activate this tag to get debugging information, see onyx.util.debugprint.DebugPrint
Return a graph for this processor. By default this is just a single node whose label is the label of processor; derived classes may wish to override this property.
Return a label for this processor. By default this is just the name of the class; derived classes may wish to override this property by providing a different label to __init__().
Process a line from an SCP file, generate an MlfBlock Event from the filename, and call the sendee with it
Internal function that pushes result into the sendee. Implementations of process() must call this to push results. To set up the sendee, (the target of the push), clients of the processor must either initialize the object with a sendee, or call set_sendee(). Processors created with a sendee of False will never send, but will not error if send is called.
The callable this processor will use to send events; see set_sendee()
Whether this processor will currently send events at all; see set_sending()
Clients call this to set up the callable where the processor will send its results.
Clients call this to turn sending from a processor on or off.
Open an HTK scp file, then open all the audio files it refers to.