Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

review the table schema spec and ensure all requirements appear in the tests #31

Merged
merged 4 commits into from
Jul 13, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 81 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,58 @@ A utility library for working with [Table Schema](https://specs.frictionlessdata
$ composer require frictionlessdata/tableschema
```

### Table

Table class allows to iterate over data conforming to a table schema

Instantiate a Table object based on a data source and a table schema.

```php
use frictionlessdata\tableschema\Table;

$table = new Table("tests/fixtures/data.csv", ["fields" => [
["name" => "first_name"],
["name" => "last_name"],
["name" => "order"]
]]);
```

Schema can be any parameter valid for the Schema object (See below), so you can use a url or filename which contains the schema

```php
$table = new Table("tests/fixtures/data.csv", "tests/fixtures/data.json");
```

iterate over the data, all the values are cast and validated according to the schema

```php
foreach ($table as $row) {
print($row["order"]." ".$row["first_name"]." ".$row["last_name"]."\n");
};
```

validate function will validate the schema and get some sample of the data itself to validate it as well

```php
Table::validate(new CsvDataSource("http://invalid.data.source/"), $schema);
```

You can instantiate a table object without schema, in this case the schema will be inferred automatically based on the data

```php
$table = new Table("tests/fixtures/data.csv");
$table->schema()->fields(); // ["first_name" => StringField, "last_name" => StringField, "order" => IntegerField]
```

Additional methods and functionality

```php
$table->headers() // ["first_name", "last_name", "order"]
$table->save("output.csv") // iterate over all the rows and save the to a csv file
$table->schema() // get the Schema object
$table->read() // returns all the data as an array
```

### Schema

Schema class provides helpful methods for working with a table schema and related data.
Expand All @@ -26,7 +78,7 @@ Schema class provides helpful methods for working with a table schema and relate

Schema objects can be constructed using any of the following:

* php array
* php array (or object)
```php
$schema = new Schema([
'fields' => [
Expand Down Expand Up @@ -68,14 +120,7 @@ $schema->missingValues(); // [""]
$schema->primaryKey(); // ["id"]
$schema->foreignKeys(); // []
$schema->fields(); // ["id" => IntegerField, "name" => StringField]
$field = $schema->field("id");
$field("id")->format(); // "default"
$field("id")->name(); // "id"
$field("id")->type(); // "integer"
$field("id")->constraints(); // (object)["required"=>true, "minimum"=>1, "maximum"=>500]
$field("id")->enum(); // []
$field("id")->required(); // true
$field("id")->unique(); // false
$field = $schema->field("id"); // Field object (See Field reference below)
```

validate function accepts the same arguemnts as the Schema constructor but returns a list of errors instead of raising exceptions
Expand Down Expand Up @@ -123,7 +168,7 @@ $schema->fields([
]);
```

appropriate field object is created according to the given descriptor
appropriate Field object is created according to the given descriptor (see below for Field class reference)

```php
$schema->field("id"); // IntegerField object
Expand All @@ -145,62 +190,52 @@ $schema->primaryKey(["id"]);

after every change - schema is validated and will raise Exception in case of validation errors

finally, save the schema to a json file
Finally, you can get the full validated descriptor

```php
$schema->save("my-schema.json");
$schema->fullDescriptor();
```

### Table

Table class allows to iterate over data conforming to a table schema

Instantiate a Table object based on a data source and a table schema.
And, save it to a json file

```php
use frictionlessdata\tableschema\Table;

$table = new Table("tests/fixtures/data.csv", ["fields" => [
["name" => "first_name"],
["name" => "last_name"],
["name" => "order"]
]]);
$schema->save("my-schema.json");
```

Schema can be any parameter valid for the Schema object, so you can use a url or filename which contains the schema

```php
$table = new Table("tests/fixtures/data.csv", "tests/fixtures/data.json");
```
### Field

iterate over the data, all the values are cast and validated according to the schema
Field class represents a single table schema field descriptor

```php
foreach ($table as $row) {
print($row["order"]." ".$row["first_name"]." ".$row["last_name"]."\n");
};
```
Create a field from a descriptor

validate function will validate the schema and get some sample of the data itself to validate it as well

```php
Table::validate(new CsvDataSource("http://invalid.data.source/"), $schema);
use frictionlessdata\tableschema\Fields\FieldsFactory;
$field = FieldsFactory::field([
"name" => "id", "type" => "integer",
"constraints" => ["required" => true, "minimum" => 5]
]);
```

You can instantiate a table object without schema, in this case the schema will be inferred automatically based on the data
Cast and validate values using the field

```php
$table = new Table("tests/fixtures/data.csv");
$table->schema()->fields(); // ["first_name" => StringField, "last_name" => StringField, "order" => IntegerField]
$field->castValue("3"); // exception: value is below minimum
$field->castValue("7"); // 7
```

Additional methods and functionality
Additional method to access field data

```php
$table->headers() // ["first_name", "last_name", "order"]
$table->save("output.csv") // iterate over all the rows and save the to a csv file
$table->schema() // get the Schema object
$table->read() // returns all the data as an array
$field("id")->format(); // "default"
$field("id")->name(); // "id"
$field("id")->type(); // "integer"
$field("id")->constraints(); // (object)["required"=>true, "minimum"=>1, "maximum"=>500]
$field("id")->enum(); // []
$field("id")->required(); // true
$field("id")->unique(); // false
$field("id")->title(); // "Id" (or null if not provided in descriptor)
$field("id")->description(); // "The ID" (or null if not provided in descriptor)
$field("id")->rdfType(); // "http://schema.org/Thing" (or null if not provided in descriptor)
```

## Important Notes
Expand Down
40 changes: 26 additions & 14 deletions src/Fields/BaseField.php
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,21 @@ public function name()
return $this->descriptor()->name;
}

public function title()
{
return isset($this->descriptor()->title) ? $this->descriptor()->title : null;
}

public function description()
{
return isset($this->descriptor()->description) ? $this->descriptor()->description : null;
}

public function rdfType()
{
return isset($this->descriptor()->rdfType) ? $this->descriptor()->rdfType : null;
}

public function format()
{
return isset($this->descriptor()->format) ? $this->descriptor()->format : 'default';
Expand Down Expand Up @@ -292,30 +307,27 @@ protected function checkMaximumConstraint($val, $maxConstraint)
return $val <= $maxConstraint;
}

protected function checkMinLengthConstraint($val, $minLength)
protected function getLengthForConstraint($val)
{
if (is_string($val)) {
return strlen($val) >= $minLength;
return strlen($val);
} elseif (is_array($val)) {
return count($val) >= $minLength;
return count($val);
} elseif (is_object($val)) {
return count($val) >= $minLength;
return count((array) $val);
} else {
throw $this->getValidationException('invalid value for minLength constraint', $val);
throw $this->getValidationException('invalid value for length constraint', $val);
}
}

protected function checkMinLengthConstraint($val, $minLength)
{
return $this->getLengthForConstraint($val) >= $minLength;
}

protected function checkMaxLengthConstraint($val, $maxLength)
{
if (is_string($val)) {
return strlen($val) <= $maxLength;
} elseif (is_array($val)) {
return count($val) <= $maxLength;
} elseif (is_object($val)) {
return count($val) <= $maxLength;
} else {
throw $this->getValidationException('invalid value for maxLength constraint', $val);
}
return $this->getLengthForConstraint($val) <= $maxLength;
}

protected function getAllowedValues()
Expand Down
2 changes: 1 addition & 1 deletion src/Fields/FieldsFactory.php
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ class FieldsFactory
/**
* get a new field object in the correct type according to the descriptor.
*
* @param object $descriptor
* @param object|array $descriptor
*
* @return BaseField
*
Expand Down
4 changes: 4 additions & 0 deletions src/Fields/GeojsonField.php
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,11 @@ protected function validateCastValue($val)
} catch (\Exception $e) {
throw $this->getValidationException($e->getMessage(), $val);
}
if (!$val) {
throw $this->getValidationException('failed to decode json', $val);
}
}
$val = json_decode(json_encode($val));
if (!is_object($val)) {
throw $this->getValidationException('must be an object', $val);
}
Expand Down
40 changes: 20 additions & 20 deletions src/Fields/GeopointField.php
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,25 @@ protected function validateCastValue($val)
if (!is_string($val)) {
throw $this->getValidationException('value must be a string', $val);
} else {
return $this->getNativeGeopoint(explode(',', $val));
$val = explode(',', $val);
if (count($val) != 2) {
throw $this->getValidationException('value must be a string with 2 comma-separated elements', $val);
} else {
return $this->getNativeGeopoint($val);
}
}
case 'array':
if (!is_array($val)) {
throw $this->getValidationException('value must be an array', $val);
if (!is_array($val) || array_keys($val) != [0, 1]) {
throw $this->getValidationException('value must be an array with 2 elements', $val);
} else {
return $this->getNativeGeopoint($val);
}
case 'object':
if (!is_object($val)) {
throw $this->getValidationException('value must be an object', $val);
} elseif (!isset($val->lat) || !isset($val->lon)) {
$val = json_decode(json_encode($val), true);
if (!is_array($val) || !array_key_exists('lat', $val) || !array_key_exists('lon', $val)) {
throw $this->getValidationException('object must contain lon and lat attributes', $val);
} else {
return $this->getNativeGeopoint([$val->lon, $val->lat]);
return $this->getNativeGeopoint([$val['lon'], $val['lat']]);
}
default:
throw $this->getValidationException('invalid format', $val);
Expand All @@ -46,20 +50,16 @@ public static function type()

protected function getNativeGeopoint($arr)
{
if (count($arr) != 2) {
throw $this->getValidationException('lon,lat array must contain only lon,lat', json_encode($arr));
list($lon, $lat) = $arr;
$lon = (int) $lon;
$lat = (int) $lat;
if (
$lon > 180 || $lon < -180
|| $lat > 90 or $lat < -90
) {
throw $this->getValidationException('invalid lon,lat values', json_encode($arr));
} else {
list($lon, $lat) = $arr;
$lon = (int) $lon;
$lat = (int) $lat;
if (
$lon > 180 || $lon < -180
|| $lat > 90 or $lat < -90
) {
throw $this->getValidationException('invalid lon,lat values', json_encode($arr));
} else {
return [$lon, $lat];
}
return [$lon, $lat];
}
}
}
17 changes: 17 additions & 0 deletions tests/FieldTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,23 @@ public function testCastValue()
$this->assertEquals(1, FieldsFactory::field($this->DESCRIPTOR_MAX)->castValue('1'));
}

public function testAdditionalMethods()
{
$field = FieldsFactory::field(['name' => 'name', 'type' => 'string']);
$this->assertEquals(null, $field->title());
$this->assertEquals(null, $field->description());
$this->assertEquals(null, $field->rdfType());
$field = FieldsFactory::field([
'name' => 'name', 'type' => 'string',
'title' => 'Title',
'description' => 'Description',
'rdfType' => 'http://schema.org/Thing',
]);
$this->assertEquals('Title', $field->title());
$this->assertEquals('Description', $field->description());
$this->assertEquals('http://schema.org/Thing', $field->rdfType());
}

public function testCastValueConstraintError()
{
try {
Expand Down
Loading