avro-scheme generates untyped Cascading Fields objects

[ldcasillas-progreso]

The Fields class in Cascading 2.2 and later supports optional field type declarations. However, when I tried to use avro-scheme with an application that makes use of this feature, it failed. The reason is a bit hard to explain, but this is the general idea:

1. When Cascading encounters a mix of typed and untyped Fields in a flow it will generally drop the field types.
2. The AvroScheme generates an untyped Fields object.
3. Therefore, the tuples sourced from the scheme cause pipes to fail downstream if they rely on the field types facility for type coercion.

For more background, see these links:

• http://docs.cascading.org/cascading/2.6/userguide/html/ch07.html
• https://groups.google.com/forum/#!searchin/cascading-user/field$20types/cascading-user/W81qPnRQS6k/RSyEh4cHWAIJ
• https://gist.github.com/ldcasillas-progreso/beff04d2b15d010bdf2c

I have started a branch of avro-scheme that generates typed fields:

• https://github.com/ldcasillas-progreso/cascading.avro/tree/field_types-2.6

I don't feel that this is ready for a merge just yet, because:

1. I started my work on the version-2.5 branch and I haven't yet digested the type conversion logic going on in version-2.6 and how this might affect what I'm doing.
2. I haven't yet thought out how to unit test these field typing changes

Yet I think it's a good time already to start the conversation on how to move forward from here on getting this feature merged upstream.

See also the discussion under issue #27 for other potential expansions of this feature.

[kkrugler]

A first step would be to define a good test for merging a Pipe sourced from Avro with a pipe that has typed fields, and ensure that we don't wind up with untyped fields downstream. One way might be to have a custom function that checks field types for the tuples it receives.