Skip to content

Commit

Permalink
Merge pull request #31 from OriHoch/master
Browse files Browse the repository at this point in the history 
review the table schema spec and ensure all requirements appear in the tests
  • Loading branch information
@OriHoch
OriHoch authored Jul 13, 2017
2 parents b38ee54 + 5bdc84d commit e46c3b8
Show file tree
Hide file tree
Showing 8 changed files with 361 additions and 211 deletions.
127 changes: 81 additions & 46 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,58 @@ A utility library for working with [Table Schema](https://specs.frictionlessdata
$ composer require frictionlessdata/tableschema
```

### Table

Table class allows to iterate over data conforming to a table schema

Instantiate a Table object based on a data source and a table schema.

```php
use frictionlessdata\tableschema\Table;

$table = new Table("tests/fixtures/data.csv", ["fields" => [
["name" => "first_name"],
["name" => "last_name"],
["name" => "order"]
]]);
```

Schema can be any parameter valid for the Schema object (See below), so you can use a url or filename which contains the schema

```php
$table = new Table("tests/fixtures/data.csv", "tests/fixtures/data.json");
```

iterate over the data, all the values are cast and validated according to the schema

```php
foreach ($table as $row) {
print($row["order"]." ".$row["first_name"]." ".$row["last_name"]."\n");
};
```

validate function will validate the schema and get some sample of the data itself to validate it as well

```php
Table::validate(new CsvDataSource("http://invalid.data.source/"), $schema);
```

You can instantiate a table object without schema, in this case the schema will be inferred automatically based on the data

```php
$table = new Table("tests/fixtures/data.csv");
$table->schema()->fields(); // ["first_name" => StringField, "last_name" => StringField, "order" => IntegerField]
```

Additional methods and functionality

```php
$table->headers() // ["first_name", "last_name", "order"]
$table->save("output.csv") // iterate over all the rows and save the to a csv file
$table->schema() // get the Schema object
$table->read() // returns all the data as an array
```

### Schema

Schema class provides helpful methods for working with a table schema and related data.
Expand All @@ -26,7 +78,7 @@ Schema class provides helpful methods for working with a table schema and relate

Schema objects can be constructed using any of the following:

* php array
* php array (or object)
```php
$schema = new Schema([
'fields' => [
Expand Down Expand Up @@ -68,14 +120,7 @@ $schema->missingValues(); // [""]
$schema->primaryKey(); // ["id"]
$schema->foreignKeys(); // []
$schema->fields(); // ["id" => IntegerField, "name" => StringField]
$field = $schema->field("id");
$field("id")->format(); // "default"
$field("id")->name(); // "id"
$field("id")->type(); // "integer"
$field("id")->constraints(); // (object)["required"=>true, "minimum"=>1, "maximum"=>500]
$field("id")->enum(); // []
$field("id")->required(); // true
$field("id")->unique(); // false
$field = $schema->field("id"); // Field object (See Field reference below)
```

validate function accepts the same arguemnts as the Schema constructor but returns a list of errors instead of raising exceptions
Expand Down Expand Up @@ -123,7 +168,7 @@ $schema->fields([
]);
```

appropriate field object is created according to the given descriptor
appropriate Field object is created according to the given descriptor (see below for Field class reference)

```php
$schema->field("id"); // IntegerField object
Expand All @@ -145,62 +190,52 @@ $schema->primaryKey(["id"]);

after every change - schema is validated and will raise Exception in case of validation errors

finally, save the schema to a json file
Finally, you can get the full validated descriptor

```php
$schema->save("my-schema.json");
$schema->fullDescriptor();
```

### Table

Table class allows to iterate over data conforming to a table schema

Instantiate a Table object based on a data source and a table schema.
And, save it to a json file

```php
use frictionlessdata\tableschema\Table;

$table = new Table("tests/fixtures/data.csv", ["fields" => [
["name" => "first_name"],
["name" => "last_name"],
["name" => "order"]
]]);
$schema->save("my-schema.json");
```

Schema can be any parameter valid for the Schema object, so you can use a url or filename which contains the schema

```php
$table = new Table("tests/fixtures/data.csv", "tests/fixtures/data.json");
```
### Field

iterate over the data, all the values are cast and validated according to the schema
Field class represents a single table schema field descriptor

```php
foreach ($table as $row) {
print($row["order"]." ".$row["first_name"]." ".$row["last_name"]."\n");
};
```
Create a field from a descriptor

validate function will validate the schema and get some sample of the data itself to validate it as well

```php
Table::validate(new CsvDataSource("http://invalid.data.source/"), $schema);
use frictionlessdata\tableschema\Fields\FieldsFactory;
$field = FieldsFactory::field([
"name" => "id", "type" => "integer",
"constraints" => ["required" => true, "minimum" => 5]
]);
```

You can instantiate a table object without schema, in this case the schema will be inferred automatically based on the data
Cast and validate values using the field

```php
$table = new Table("tests/fixtures/data.csv");
$table->schema()->fields(); // ["first_name" => StringField, "last_name" => StringField, "order" => IntegerField]
$field->castValue("3"); // exception: value is below minimum
$field->castValue("7"); // 7
```

Additional methods and functionality
Additional method to access field data

```php
$table->headers() // ["first_name", "last_name", "order"]
$table->save("output.csv") // iterate over all the rows and save the to a csv file
$table->schema() // get the Schema object
$table->read() // returns all the data as an array
$field("id")->format(); // "default"
$field("id")->name(); // "id"
$field("id")->type(); // "integer"
$field("id")->constraints(); // (object)["required"=>true, "minimum"=>1, "maximum"=>500]
$field("id")->enum(); // []
$field("id")->required(); // true
$field("id")->unique(); // false
$field("id")->title(); // "Id" (or null if not provided in descriptor)
$field("id")->description(); // "The ID" (or null if not provided in descriptor)
$field("id")->rdfType(); // "http://schema.org/Thing" (or null if not provided in descriptor)
```

## Important Notes
Expand Down
40 changes: 26 additions & 14 deletions src/Fields/BaseField.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,21 @@ public function name()
return $this->descriptor()->name;
}

public function title()
{
return isset($this->descriptor()->title) ? $this->descriptor()->title : null;
}

public function description()
{
return isset($this->descriptor()->description) ? $this->descriptor()->description : null;
}

public function rdfType()
{
return isset($this->descriptor()->rdfType) ? $this->descriptor()->rdfType : null;
}

public function format()
{
return isset($this->descriptor()->format) ? $this->descriptor()->format : 'default';
Expand Down Expand Up @@ -292,30 +307,27 @@ protected function checkMaximumConstraint($val, $maxConstraint)
return $val <= $maxConstraint;
}

protected function checkMinLengthConstraint($val, $minLength)
protected function getLengthForConstraint($val)
{
if (is_string($val)) {
return strlen($val) >= $minLength;
return strlen($val);
} elseif (is_array($val)) {
return count($val) >= $minLength;
return count($val);
} elseif (is_object($val)) {
return count($val) >= $minLength;
return count((array) $val);
} else {
throw $this->getValidationException('invalid value for minLength constraint', $val);
throw $this->getValidationException('invalid value for length constraint', $val);
}
}

protected function checkMinLengthConstraint($val, $minLength)
{
return $this->getLengthForConstraint($val) >= $minLength;
}

protected function checkMaxLengthConstraint($val, $maxLength)
{
if (is_string($val)) {
return strlen($val) <= $maxLength;
} elseif (is_array($val)) {
return count($val) <= $maxLength;
} elseif (is_object($val)) {
return count($val) <= $maxLength;
} else {
throw $this->getValidationException('invalid value for maxLength constraint', $val);
}
return $this->getLengthForConstraint($val) <= $maxLength;
}

protected function getAllowedValues()
Expand Down
2 changes: 1 addition & 1 deletion src/Fields/FieldsFactory.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ class FieldsFactory
/**
* get a new field object in the correct type according to the descriptor.
*
* @param object $descriptor
* @param object|array $descriptor
*
* @return BaseField
*
Expand Down
4 changes: 4 additions & 0 deletions src/Fields/GeojsonField.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,11 @@ protected function validateCastValue($val)
} catch (\Exception $e) {
throw $this->getValidationException($e->getMessage(), $val);
}
if (!$val) {
throw $this->getValidationException('failed to decode json', $val);
}
}
$val = json_decode(json_encode($val));
if (!is_object($val)) {
throw $this->getValidationException('must be an object', $val);
}
Expand Down
40 changes: 20 additions & 20 deletions src/Fields/GeopointField.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,25 @@ protected function validateCastValue($val)
if (!is_string($val)) {
throw $this->getValidationException('value must be a string', $val);
} else {
return $this->getNativeGeopoint(explode(',', $val));
$val = explode(',', $val);
if (count($val) != 2) {
throw $this->getValidationException('value must be a string with 2 comma-separated elements', $val);
} else {
return $this->getNativeGeopoint($val);
}
}
case 'array':
if (!is_array($val)) {
throw $this->getValidationException('value must be an array', $val);
if (!is_array($val) || array_keys($val) != [0, 1]) {
throw $this->getValidationException('value must be an array with 2 elements', $val);
} else {
return $this->getNativeGeopoint($val);
}
case 'object':
if (!is_object($val)) {
throw $this->getValidationException('value must be an object', $val);
} elseif (!isset($val->lat) || !isset($val->lon)) {
$val = json_decode(json_encode($val), true);
if (!is_array($val) || !array_key_exists('lat', $val) || !array_key_exists('lon', $val)) {
throw $this->getValidationException('object must contain lon and lat attributes', $val);
} else {
return $this->getNativeGeopoint([$val->lon, $val->lat]);
return $this->getNativeGeopoint([$val['lon'], $val['lat']]);
}
default:
throw $this->getValidationException('invalid format', $val);
Expand All @@ -46,20 +50,16 @@ public static function type()

protected function getNativeGeopoint($arr)
{
if (count($arr) != 2) {
throw $this->getValidationException('lon,lat array must contain only lon,lat', json_encode($arr));
list($lon, $lat) = $arr;
$lon = (int) $lon;
$lat = (int) $lat;
if (
$lon > 180 || $lon < -180
|| $lat > 90 or $lat < -90
) {
throw $this->getValidationException('invalid lon,lat values', json_encode($arr));
} else {
list($lon, $lat) = $arr;
$lon = (int) $lon;
$lat = (int) $lat;
if (
$lon > 180 || $lon < -180
|| $lat > 90 or $lat < -90
) {
throw $this->getValidationException('invalid lon,lat values', json_encode($arr));
} else {
return [$lon, $lat];
}
return [$lon, $lat];
}
}
}
17 changes: 17 additions & 0 deletions tests/FieldTest.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,23 @@ public function testCastValue()
$this->assertEquals(1, FieldsFactory::field($this->DESCRIPTOR_MAX)->castValue('1'));
}

public function testAdditionalMethods()
{
$field = FieldsFactory::field(['name' => 'name', 'type' => 'string']);
$this->assertEquals(null, $field->title());
$this->assertEquals(null, $field->description());
$this->assertEquals(null, $field->rdfType());
$field = FieldsFactory::field([
'name' => 'name', 'type' => 'string',
'title' => 'Title',
'description' => 'Description',
'rdfType' => 'http://schema.org/Thing',
]);
$this->assertEquals('Title', $field->title());
$this->assertEquals('Description', $field->description());
$this->assertEquals('http://schema.org/Thing', $field->rdfType());
}

public function testCastValueConstraintError()
{
try {
Expand Down
Loading

0 comments on commit e46c3b8

Please sign in to comment.