-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathamie.txt
4217 lines (3344 loc) · 169 KB
/
amie.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
AMIE Notes
==========
Steven K. Baum
0.1, Dec. 9, 2021
:doctype: book
:toc:
:icons:
:source-highlighter: coderay
:numbered!:
== HPRC AMIE Implementation
=== Overview
This document explains the development of the HPRC implementation of the *Account Management Information Exchange (AMIE)* software
system that provides the capability for XSEDE to manage accounts and track resource usage among
all member institutions.
It provides a distributed accounting mechanism across all of its sites so that an individual users usage can be
tracked across multiple resources. AMIE is also responsible for propagating *Distinguished Names* across the XSEDE sites.
=== Summary for Interfacing with the Local Account Management Database
All the information specific to the local user and project information is contained within
the *local_info* database, the creation and contents of which are described in the
xref:local_info[Setting Up DB Tables] section dealing with the creation of the various tables using PostgreSQL
and the Python https://www.psycopg.org/docs/[psycopg2] interface to that database system.
That section contains a description of the columns within the *local_info* as well as the details of how it
was created.
The general flow of the local AMIE implementation system is:
* The Python script `dump_approvals.py` cycles through all queued AMIE packets, extracting
project and user information that is inserted into the *local_info* table.
The AMIE packets - e.g. `request_project_create`, `request_account_create`, etc. - that require local TAMU approval before further processing are entered
in the *approval* table. This script can and eventually will be run periodically via cron.
* The Python script `approval.py` is run on the command line to enable each non-approved request
in the *approval* table to be approved. It also provides the capability of modifying the automatically
generated local person ID.
* The Python script `respond.py`
creates and sends appropriate response packets to the NCSA for approved requests.
This script also creates and modifies entries in various database tables to track the progress
of the sequence for each packet type. This script also can and eventually will be run
periodically via cron.
=== AMIE Links
*XSEDE Research Software Portal*
https://software.xsede.org/packaged-software/amie[`https://software.xsede.org/packaged-software/amie`]
*XSEDE AMIE Software Repository*
https://software.xsede.org/production/amie/[`https://software.xsede.org/production/amie/`]
*XSEDE AMIE Github Repository*
https://github.com/xsede/amieclient/[`https://github.com/xsede/amieclient/`]
*Stanford AMIE Github Repository*
https://github.com/stanford-rc/xsede-amie-python[`https://github.com/stanford-rc/xsede-amie-python`]
=== Amie Documents
The PDF documents available for download at the *XSEDE AMIE Software Repository* are listed below.
All are contained within a tar file at that site. A significant portion of the content of these
documents has been included in this document to make access to the scattered and disparate PDF
documents easier.
*AMIE API Testing* - A 2-page overview of the seven kinds of initial packets handled by the AMIE client API.
*AMIE Client API Routes Overview* - A 4-page overview of the AMIE transaction and and packet
model, wherein the data is presented in JSON form via a RESTful API.
*AMIE in XSEDE* - A 35-page document describing the sets of AMIE transactions used by
XSEDE. It describes the sequence of packets required by each type of transaction, and also
the required and optional information for each type of packet. A familiarity with the
*AMIE Model* document is assumed. These details are available within the
xref:_transactions_originated_at_xdcdb[Transactions Originated at XDCDB] and
xref:_transactions_originated_locally[Transactions Originated Locally] sections
of this document.
*AMIE Model* - A 5-page overview of the AMIE model wherein the terms are defined,
the packet types listed, and a skeletal example is presented. The contents of this document are contained
within the xref:_amie_model[AMIE Model] section below.
*Implementing AMIE* - A 29-page document describing how to connect a local account
management system to AMIE. It describes a reference implementation database that is
an intermediate database standing between AMIE and the local account management
database. This describes the tables that need to be created for the intermediate database
using data definition language (DDL), leaving the choice of actual database to the user.
The details of the actual PostgreSQL tables created using these guidelines can be found in the
xref:_setting_up_db_tables[Setting Up DB Tables] section below. There were some minor changes
to the suggested tables as well as additional tables created for local use.
*XSEDE Usage API User's Guide* - A 14-page document that describes the RESTful API used
by an XSEDE Service Provider to report usage to XSEDE.
== AMIE Model
=== Overview
The AMIE model consists of two sites and an agreed upon set of *transactions* that the two
sites will use to send account management data to each other. A transaction consists of
*packets* of data sent between the two sites. The site sending a packet is called the *local site*
and the packet is known to the sending site as an *outgoing packet*. The receiving site is
called the *remote site* and the packet received is known to the receiving site as an
*incoming packet*. The site that creates the transaction (and sends the first packet) is also
called the *originating site*.
From the AMIE perspective, XSEDE consists of a number of local sites that exchange account
management information with the XSEDE Central Database (XDCDB).
This document is written from the perspective of a local site. The remote site is thus the XSEDE
Central Database (XDCDB). Incoming packets are packets sent by XDCDB to the local site.
Outgoing packets are packets sent to the XDCDB by the local site.
=== Transactions
Transactions have a number of properties. These are the *local site*, the *remote site*, the *originating
site*, a *transaction id*, and a *state*. Once created, the first four properties do not change. The *state*
changes over time.
The *transaction id* is used to distinguish one transaction from another. The originating site
chooses the *transaction id* without consulting the remote site. The only rule is that a transaction id
created by one site may not be reused by that site for a different transaction. Hence a transaction is
identified by the *originating site*, the *transaction id*, the *local site*, and the *remote site*.
AMIE defines 3 states for a transaction: *in-progress*, *completed*, or *failed*. The initial state of a
transaction is *in-progress*. It remains in that state until all packets have been processed. If all
packets have been successfully processed, the transaction state becomes *completed*. If any of the
packets causes a failure, the transaction state becomes *failed*.
=== Packets
Transactions also have packets which contain account management data. *Incoming packets* are
those packets received from the remote site. *Outgoing packets* are those packets created by the
local site to be sent to the remote site. Outgoing packets are created either when the transaction is
created or as a reply to an incoming packet.
AMIE does not specify a pre-defined set of transactions. It specifies a set of packets which can be
used within transactions, but the sites must agree on the packets that are used within transactions
as well as the ordering of those packets.
A packet has a number of properties. These are *type*, *version*, *packet id*, and *state*. It also has a
list of *expected replies*.
Each packet has a *packet id* which is chosen by the site that creates the packet. It has to be unique
within the set of *outgoing packets* for a given transaction3.
Each packet has a *state*. AMIE defines 3 states for a packet: *in-progress*, *completed*, or *failed*.
Each packet must also provide a list of *expected replies*. An expected reply is the type of packet
that a site expects to receive from the remote site after the remote site has processed the packet.
The expected reply list allows sites to determine which packets will be part of a transaction and the
ordering of the packets within the transaction, rather than this being dictated by AMIE.
=== Transaction Packet Sequences
Each packet not only has data associated with it, but also, in most cases, an expected reply. The
expected reply mechanism is used by AMIE implementations to specify packet sequences within
transactions. For example, a `request_project_create` packet could specify that it expects a
`notify_project_create` packet in reply, which could specify that it expects a `data_project_create`
packet, which could specify that it expects an `inform_transaction_complete` packet in reply. The
`inform_transaction_complete` packet is the only packet which can not have an expected reply. It
would complete the sequence (and the transaction).
=== Types of Transactions
These are the three categories of transactions, all of which link to the appropriate sections below.
* xref:_transactions_originated_at_xdcdb[transactions that originate from XDCDB],
* xref:_transactions_originated_locally[transactions that originate locally], and
* xref:_duplicate_people_transactions[duplicate people transactions].
These are the transactions that originate from XDCDB, with all linked to their explanatory sections.
* xref:_request_project_create_transaction[`request_project_create`]
* xref:_request_account_create_transaction[`request_account_create`]
* xref:_request_project_inactivate_transaction[`request_project_inactivate`]
* xref:_request_project_reactivate_transaction[`request_project_reactivate`]
* xref:_request_account_inactivate_transaction[`request_account_inactivate`]
* xref:_request_account_reactivate_transaction[`request_account_reactivate`]
* xref:_request_user_modify_transaction[`request_user_modify`]
These are the transactions that originate locally, with all linked to their explanatory sections.
* xref:_notify_project_create_transaction[`notify_project_create`]
* xref:_notify_account_create_transaction[`notify_account_create`]
* xref:_notify_project_inactivate_transaction[`notify_project_inactivate`]
* xref:_notify_project_reactivate_transaction[`notify_project_reactivate`]
* xref:_notify_account_inactivate_transaction[`notify_account_inactivate`]
* xref:_notify_account_reactivate_transaction[`notify_account_reactivate`]
* xref:_notify_project_usage_transaction[`notify_project_usage`]
* xref:_notify_user_modify_transaction[`notify_user_modify`]
These are the duplicate people transactions, with all linked to their explanatory sections.
* xref:_notify_person_duplicate_transaction[`notify_person_duplicate`]
* xref:_request_person_merge_transaction[`request_person_merge`]
* xref:_notify_person_ids_transaction[`notify_person_ids`]
=== Transactions Originated at XDCDB
Transactions that originate at the XDCDB are requests made upon the local sites to create projects
and accounts, or requests to modify information about a project, account, or a user. Often this will
mean that an event occurred at one site that the XDCDB then asks the other sites to also perform.
For example, one site creates a project and notifies the XDCDB about the project (using a
transaction that originates at the local site). The XDCDB then would send requests to all of the
other sites to create that project at their site (using transactions that originate at the XDCDB).
For these types of transactions, local sites do not have to create a transaction. AMIE does this for
them. Local sites need to process incoming packets and create the appropriate outgoing packets in
reply to the incoming packets (including the packet data and the expected reply).
==== `request_project_create` Transaction
The purpose of this transaction is to ask a local site to create a project and an account for the PI of
the project.
The sequence of actions/packets is:
. The XDCDB sends the project and account information in a `request_project_create`
packet to the local site.
. The local site:
.. reads the `request_project_create` packet,
.. creates the local project and account
information, and then
creates entries in the *local_info* table in the AMIE database,
* `pi_person_id` - a locally generated ID string for the PI of a project
* `project_id` - a locally generated ID string for the project
* `remote_site_login` - a locally generated login name for the PI of a project
* `grant_number` - the grant number of the project
.. creates entries for the *transaction_tbl* table in the AMIE database from the information in the packet,
* `trans_rec_id` - a unique ID number for a given transaction of any of the available transaction types listed in *type_des*
* `originating_site_name` - the site from which the packet originates
* `transaction_id` - an ID number chosen by the originating site that is used to distinguish on transaction from another
* `local_site_name` - the site that sends the first packet in a transaction
* `remote_site_name` - the site that receives the first packet in a transaction
* `state_id` - a foreign key into the *state_des* table
* `ts` - an automatically created time stamp
.. creates entries for the *packet_tbl* table in the AMIE database from the information in the packet,
* `packet_rec_id` - a unique ID number for a packet within a transaction for which there may be multiple packets
* `trans_rec_id` - a unique ID number for a given transaction of any of the available transaction types listed in *type_des*
* `packet_id` - the ID for a packet as specified by AMIE
* `type_id` - a foreign key into the *type_des* table that specifies the packet type
* `version` - the version of AMIE used for this packet
* `state_id` - a foreign key into the *state_des* table that specifies the packet state
* `outgoing_flag` - 1 for and outgoing and 0 for an incoming packet
* `ts` - an auto-generated timestamp
.. creates entries for the *data_tbl* table in the AMIE database from the information in the packet,
* `packet_rec_id` - a unique ID number for a packet within a transaction for which there may be multiple packets
* `tag` - the key for an entry in the dictionary in the `body` portion of the JSON packet
* `subtag` - either `NULL` or the key for an entry in a dictionary that is the value for a 'body' key
* `seq` - a sequence number for key values, which is 0 for single values and a sequence number starting at 0 for values that are lists or dictionaries with multiple entries
* `value` - the value corresponding to the `tag` key in the dictionary
.. creates entries for the *transaction_depends_tbl*
* `trans_rec_id` - a foreign key into *transaction_tbl* that references the transaction that has dependencies
* `depends_on_trans_rec_id` - a foreign key into *transaction_tbl* that references a transaction that must be completed
before the `trans_rec_id` transaction can be processed
.. sends the site-specific project and
account information back to the XDCDB in a `notify_project_create` packet.
. The XDCDB
sends additional information about the project to the local site in a `data_project_create` packet.
. The local site finishes the transaction by sending an `inform_transaction_complete`.
The sequence of packets is:
[width=50%]
[cols="2,1"]
|===
| `request_project_create` | incoming
| `notify_project_create` | outgoing
| `data_project_create` | incoming
| `inform_transaction_complete` | outgoing
|===
===== `request_project_create` Packet
The `request_project_create` packet contains information to be used by a local site to create a
project and an account for the PI on the project. The result of this request should be the creation of
a local site project (with a local project ID) as well as an account for the PI (with a local person
ID).
*NOTE*: A `request_project_create` implies project reactivation. If the project had been inactivated
(by a `request_project_inactivate`), the site must reactivate the project as well as performing the
other actions required by this transaction. The only account that should be reactivated would be the
account for the PI in the `request_project_create`.
*NOTE*: The `PiGlobalID` is the internal ID for the PI within the XDCDB. It could be used to
uniquely identify the PI.
===== Locally Created Information
Upon reception of a `notify_project_create` packet a local site needs to create values for
the `ProjectID`, `PiPersonID` and `PiRemoteSiteLogin` tags for the return packet, as well as for identification
purposes with future packet transactions involving the same project.
At TAMU HPRC the values for these tags are created via the following procedures:
* `ProjectID` - the concatenation of 'p.', the `GrantNumber` tag value in lower case, and '.000'.
* `PiPersonID` - the concatenation of 'pi.', the `FirstName` first letter in lower case, the `LastName` first letter
in lower case, and the `PiGlobalID` value
* `PiRemoteSiteLogin` - this is set as identical to the `PiPersonID`
===== `notify_project_create` Packet
After having created a project and an account, the local site must reply with a
`notify_project_create` packet. *This packet must contain*:
* the local project ID, i.e. *ProjectID*,
* the local ID for the PI, i.e. *PiPersonID*,
which should be a locally unique identifier for the PI at the local site,
* and the login at the local site
for the PI, i.e. *PiRemoteSiteLogin*
* the *GrantNumber* from the
`request_project_create` packet,
The `ResourceList` will only have one resource (and it will be a
resource at the remote site).
The following table lists the tags that can be included in the `notify_project_create` packet, although
the only required tags are in *bold*.
[[allocationtype]]
The `AllocationType` must be one of the following:
* `new` - Creates a new project and creates a new allocation on a specific resource.
* `renewal` - A continuing award for an existing project.
* `supplement` - Adds SUs to an existing allocation (without changing the start or end dates).
* `extension` - Extends the end date of an allocation. An extension should only be received
for an existing allocation.
* `transfer` - Occurs as one of a pair of transactions – one negative, and one positive. A
negative transfer for an unknown allocation is an error. Otherwise, the amount is deducted
from the allocation. A positive transfer should be treated the same as a supplement.
* `advance` - An allocation given in anticipation of a new or renewal allocation. It should
be treated as a supplement.
* `adjustment` - Modifies an existing allocation for other reasons. It should be handled as a
transfer type; but unlike a transfer, there is only one transaction.
.Allowable Tags for `notify_project_create` Packet
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *GrantNumber* | | | AST040002
| *PfosNumber* | | | 120
| *PiOrgCode* | | | 0017756
| *PiPersonID* | | | 6751
| *PiRemoteSiteLogin* | | | squinn
| *ProjectID* | | | 1001
| *ProjectTitle* | | | Planetary Motion
| *ResourceList* | | | this_resource
| *ServiceUnitsAllocated* | | | 99999
| *StartDate* | | | 2003-12-16
| `Abstract` | | | Some text...
| `AcademicDegree` | *Degree* | 0 | MS
| `AcademicDegree` | *Field* | 0 | Math
| `AcademicDegree` | *Degree* | 1 | PhD
| `AcademicDegree` | *Field* | 1 | Math
| xref:allocationtype[`AllocationType`] | | |
| `EndDate` | | | 2013-12-31
| `NsfStatusCode` | | | CN
| `PiBusinessPhoneComment` | | | Business phone on campus
| `PiBusinessPhoneExtension` | | | 7570
| `PiBusinessPhoneNumber` | | | (217) 867-5309
| `PiCity` | | | Champaign
| `PiCountry` | | | USA
| `PiDepartment` | | | N.C.S.A.
| `PiDnList` | | 0 | /C=US/O=NCSA/CN=Steven James Quinn
| `PiDnList` | | 1 | /C=US/O=PSC/CN=Steven James Quinn
| `PiDnList` | | 2 | /C=US/O=SDSC/CN=Steven James Quinn
| `PiFax` | | | (217) 555-5555
| `PiFirstName` | | | Steven
| `PiGlobalID` | | | 70
| `PiHomePhoneComment` | | | Has caller ID
| `PiHomePhoneExtension` | | | none
| `PiHomePhoneNumber` | | | (217) 222-0000
| `PiLastName` | | | Quinn
| `PiMiddleName` | | | James
|===
===== `notify_project_create` Section of `process_packets.py`
-----
if packet_type == 'request_project_create':
grant_number = packet.GrantNumber
record_id = packet.RecordID
project_id = packet.ProjectID # site project_id (if known)
resource = packet.ResourceList[0] # xsede site resource name, eg, delta.ncsa.xsede.org
request_type = packet.RequestType
allocation_type = packet.AllocationType # new, renewal, supplement, transfer, adjustment, advance, extension, ...
start_date = packet.StartDate
end_date = packet.EndDate
amount = packet.ServiceUnitsAllocated
abstract = packet.Abstract
project_title = packet.ProjectTitle
board_type = packet.BoardType
pfos_num = packet.PfosNumber
pi_person_id = packet.PiPersonID # site person_id for the PI (if known)
pi_first_name = packet.PiFirstName
pi_middle_name = packet.PiMiddleName
pi_last_name = packet.PiLastName
pi_organization = packet.PiOrganization
pi_department = packet.PiDepartment
pi_email = packet.PiEmail
pi_phone_number = packet.PiBusinessPhoneNumber
pi_phone_extension = packet.PiBusinessPhoneExtension
pi_address1 = packet.PiStreetAddress
pi_address2 = packet.PiStreetAddress2
pi_city = packet.PiCity
pi_state = packet.PiState
pi_zipcode = packet.PiZip
pi_country = packet.PiCountry
pi_nsf_status_code = packet.NsfStatusCode
pi_requested_logins = packet.PiRequestedLoginList
pi_dn_list = packet.PiDnList
# SP:
# - add code to find the PI from the local database (or create the person in the local database)
# and set pi_person_id, pi_login
# - add code to create the project for the grant_number (if project doesn't exist), or apply the action specified by allocation_type
# - set the project_id to the local id for the project (if it isn't already set from the RPC)
# - set the project state to active (if it is inactive), as the XDCDB will not send RPCs for inactive projects
#
# NOTE: if the record_id is not null, you should track it (associate it with the packet_rec_id).
# If a second RPC gets sent with the same record_id, the second RPC should not be processed,
# but the data from the first RPC sent in the reply NPC
# construct a NotifyProjectCreate(NPC) packet.
npc = packet.reply_packet()
npc.ProjectID = project_id # local project ID
npc.PiPersonID = pi_person_id # local person ID for the pi
# send the NPC
amie_client.send_packet(npc)
-----
===== `data_project_create` Packet
The XDCDB will reply with a data_project_create packet that contains all the DNs known by the
XDCDB for the project PI.
This packet will have the form:
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| `DnList` | | 0 | /C=US/O=NCSA/CN=Steven James Quinn
| `DnList` | | 1 | /C=US/O=PSC/CN=Steven James Quinn
| `DnList` | | 2 | /C=US/O=SDSC/CN=Steven James Quinn
| *PersonID* | | | 6751
| *ProjectID* | | | afm
|===
===== `data_project_create` Section of `process_packets.py`
-----
if packet_type == 'data_project_create':
person_id = packet.PersonID
project_id = packet.ProjectID
dn_list = packet.DnList
# the data_project_create(DPC) packet has two functions:
# 1. to let the site know that the project and PI account have been setup in the XDCDB
# 2. to provide any new DNs for the PI that were added after the RPC was sent
# NOTE: a DPC does *not* have the resource. You have to get the resource from the RPC for the trans_rec_id
# construct the InformTransactionComplete(ITC) success packet
itc = packet.reply_packet()
itc.StatusCode = 'Success'
itc.DetailCode = '1'
itc.Message = 'OK'
# send the ITC
amie_client.send_packet(itc)
-----
===== `inform_transaction_complete` Packet
The local site must then reply with an `inform_transaction_complete` to complete the transaction.
==== `request_account_create` Transaction
The purpose of this transaction is to ask a local site to create an account for a user on a project. The
XDCDB sends the account information in a `request_account_create` packet. The local site creates
the account and sends the site-specific account information back to the XDCDB in a
`notify_account_create` packet. The XDCDB then sends additional information about the account to
the local site in a `data_account_create` packet. The local site finishes the transaction by sending an
`inform_transaction_complete`.
[width=50%]
[cols="2,1"]
|===
| `request_account_create` | incoming
| `notify_acccount_create` | outgoing
| `data_account_create` | incoming
| `inform_transaction_complete` | outgoing
|===
Notes about edge cases that must eventually be dealt with:
* Although the XDCDB will not send a `request_account_create` packet before it has sent a
corresponding `request_project_create` packet, it may send it before the `request_project_create`
transaction has completed. Local sites must be prepared to delay processing of
`request_account_create` packets until they have completed the corresponding
`request_project_create` transaction.
* If the project is active, a `request_account_create` implies account reactivation. If the
account had been inactivated (by a `request_account_inactivate`), the site must reactivate the
account as well as performing the other actions required by this transaction. The site will not
receive a request_account_reactivate in addition to this transaction.
* If the project is inactive, a `request_account_create` may be sent to add a user to the project.
The user should be added as inactive. A `request_account_inactivate` will follow upon completion
of this transaction.
===== `request_account_create` Packet
The `request_account_create` packet contains the `GrantNumber` for the related project as well as the
information about the user for whom the account is to be created. If the XDCDB has been notified
of the local project ID in a previous `request_project_create` transaction, then the `ProjectID` will be
the local site project ID. Otherwise the `ProjectID` will not be present. This implies that sites must
maintain a mapping between local projects and the `GrantNumber`.
*NOTE*: The `UserGlobalID` is the internal ID for the user within the XDCDB. It could be used to
uniquely identify the user.
The `request_account_create` packet contains the following entries. The *bold* entries are required.
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *GrantNumber* | | 0 | AST040002
| *ResourceList* | | 0 | oh.ncsa.xsede
| `SitePersonID` | *Site* | 0 | ANL
| `SitePersonID` | *PersonID* | 0 | mshapiro
| `SitePersonID` | *Site* | 1 | SDSC
| `SitePersonID` | *PersonID* | 1 | 22998
| `AcademicDegree` | *Degree* | 0 | MS
| `AcademicDegree` | *Field* | 0 | Math
| `AcademicDegree` | *Degree* | 1 | PhD
| `AcademicDegree` | *Field* | 1 | Math
| *UserFirstName* | | | Michael
| *UserLastName* | | | Shapiro
| *UserOrganization* | | | UIUC
| *UserOrgCode* | | | 0017756
| `NsfStatusCode` | | | CN
| `ProjectID` | | | afm
| ... | | |
|===
The `GrantNumber` must be used to tie the request_acount_create to the corresponding project since the `ProjectID`
may not be present.
===== Typical `notify_account_create` Packet Contents
The header typically contains:
-----
expected_reply_list -> [{'type': 'notify_account_create', 'timeout': 30240}]
packet_id -> 1
trans_rec_id -> 111135727
transaction_id -> 3
remote_site_name -> TAMU
local_site_name -> TGCDB
originating_site_name -> TGCDB
outgoing_flag -> 1
packet_rec_id -> 222152621
packet_timestamp -> 2022-03-09T20:29:45.177Z
client_state -> None
packet_state -> in-progress
client_json -> None
transaction_state -> in-progress
-----
The body typically contains:
-----
ProjectID -> TAMU-52738
GrantNumber -> EVE226342
UserPasswordAccessEnable -> 1
ResourceList -> ['test-resource1.tamu.xsede']
UserRequestedLoginList -> ['tzcoolman']
AllocatedResource -> test-resource1.tamu.xsede
UserGlobalID -> 104986
UserEmail -> [email protected]
UserFirstName -> enze
UserMiddleName ->
UserLastName -> liu
UserDepartment -> Schoolf of Medicine
UserTitle ->
UserOrganization -> Indiana University
UserOrgCode -> 0087312
NsfStatusCode -> F
UserDnList -> ['/C=US/O=National Center for Supercomputing Applications/CN=enze liu 1', '/C=US/O=Pittsburgh Supercomputing Center/CN=enze liu 1']
SitePersonId -> [{'Site': 'XD-ALLOCATIONS', 'PersonID': 'tzcoolman'}, {'Site': 'X-PORTAL', 'PersonID': 'tzcoolman'}]
AcademicDegree -> [{'Degree': 'PhD', 'Field': 'Bioinformatics'}]
-----
===== `notify_account_create` Packet
After having created an account, the local site must reply with a `notify_account_create` packet.
This packet must contain:
* the local project ID `ProjectID` that has been previously created and which is included in the `request_account_create` packet,
* the local ID for the account user `UserPersonID` which must be newly created,
* the login at the local site for the user `UserRemoteSiteLogin` which must be newly created,
* the `ResourceList` which is extracted from `request_account_create` packet.
The table below lists the tags that
could be included in this packet (the ones in *bold* must be included). However, in a reply to a
`request_account_create`, if the user is already known to the XDCDB (based on the `UserPersonID`
or the `UserGlobalID`), then all the `User` tags are ignored.
The `UserGlobalID` is the internal ID for the user within the XDCDB.
The `NsfStatusCode` must be a valid NSF Status Code (as defined in the XDCDB). The
`UserOrgCode` must be a valid NSF organization code (as defined in the XDCDB). The
`UserCountry` must be a valid NSF country code (as defined in the XDCDB). The `UserState` must
be a valid state abbreviation (as defined in the XDCDB).
An originating `notify_account_create` (i.e., one that is not sent in reply to a `request_account_create`) must specify
the start date of the allocation to which the account will be associated.
The `ResourceLogin` entry is of the same form as the `AcademicDegree` entry.
-----
AcademicDegree -> [{'Degree': 'PhD', 'Field': 'Bioinformatics'}]
ResourceLogin -> [{'Resource': 'test-resource1.tamu.xsede', 'Login': 'mshapiro'}]
-----
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *ProjectID* | | | afm
| *ResourceList* | | 0 | viz.nist.xsede
| `ResourceLogin` | *Resource* | 0 | portal.teragrid
| `ResourceLogin` | *Login* | 0 | mshapiro
| `AcademicDegree` | *Degree* | 0 | MS
| `AcademicDegree` | *Field* | 0 | Math
| `AcademicDegree` | *Degree* | 1 | PhD
| `AcademicDegree` | *Field* | 1 | Math
| `NsfStatusCode` | | | CN
| `RoleList` | | 0 | co_pi
| `StartDate` | | | 2003-12-16
| `UserBusinessPhoneComment` | | | Business phone on campus
| `UserBusinessPhoneExtension` | | | 1514
| `UserBusinessPhoneNumber` | | | (217) 244-0072
| `UserCity` | | | Chammpaign
| `UserCountry` | | | 9US
| `UserDepartment` | | | N.C.S.A.
| `UserDnList` | | | /C=US/O=NCSA/CN=Michael Shapiro
| `UserDnList` | | | /C=US/O=SDSC/CN=Michael Shapiro
| `UserEmail` | | | [email protected]
| `UserFax` | | | (217)-244-1987
| `UserFirstName` | | | Michael
| ... | | |
|===
===== `notify_account_create` Section of `process_packets.py`
-----
if packet_type == 'request_account_create':
grant_number = packet.GrantNumber
project_id = packet.ProjectID # site project_id
resource = packet.ResourceList[0] # xsede site resource name, eg, delta.ncsa.xsede.org
user_person_id = packet.UserPersonID # site person_id for the User (if known)
user_login = packet.UserRemoteSiteLogin # login on resource for the User (if known)
user_first_name = packet.UserFirstName
user_middle_name = packet.UserMiddleName
user_last_name = packet.UserLastName
user_organization = packet.UserOrganization
user_department = packet.UserDepartment
user_email = packet.UserEmail
user_phone_number = packet.UserBusinessPhoneNumber
user_phone_extension = packet.UserBusinessPhoneExtension
user_address1 = packet.UserStreetAddress
user_address2 = packet.UserStreetAddress2
user_city = packet.UserCity
user_state = packet.UserState
user_zipcode = packet.UserZip
user_country = packet.UserCountry
user_nsf_status_code = packet.UserNsfStatusCode
user_requested_logins = packet.UserRequestedLoginList
user_dn_list = packet.UserDnList
# SP: add code to find the User from the local database (or create the person in the local database)
# then add an account for the User on the specified project (project_id) on the resource
# RACs are also used to reactivate accounts, so if the account already exists, just set it active
# construct a NotifyAccountCreate(NAC) packet.
nac = packet.reply_packet()
nac.ProjectID = project_id # local project ID
nac.UserRemoteSiteLogin = user_login # local login for the User on the resource
nac.UserPersonID = user_person_id # local person ID for the User
# send the NAC
amie_client.send_packet(nac)
-----
===== `data_account_create` Packet
The XDCDB will reply with a `data_account_create` packet that contains all the DNs known by the
XDCDB for the user.
The packet will contain the following:
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *PersonID* | | | 21619
| *ProjectID* | | | afm
| `DnList` | | 0 | /C=US/O=NCSA/CN=Michael Shapiro
| `DnList` | | 1 | /C=US/O=SDSC/CN=Michael Shapiro
|===
===== `inform_transaction_complete` Packet
The local site must then reply with an `inform_transaction_complete` to complete the transaction.
===== `data_account_create` Section of `process_packets.py`
-----
if packet_type == 'data_account_create':
person_id = packet.PersonID
project_id = packet.ProjectID
dn_list = packet.DnList
# the data_account_create(DAC) packet has two functions:
# 1. to let the site know that the User account on the project has been setup in the XDCDB
# 2. to provide any new DNs for the User that were added after the RAC was sent
# NOTE: a DAC does *not* have the resource. You have to get the resource from the RAC for the trans_rec_id
# construct the InformTransactionComplete(ITC) success packet
itc = packet.reply_packet()
itc.StatusCode = 'Success'
itc.DetailCode = '1'
itc.Message = 'OK'
# send the ITC
amie_client.send_packet(itc)
-----
==== `request_project_inactivate` Transaction
The purpose of this transaction is to ask a local site to inactivate a project. The XDCDB sends the
project information in a request_project_inactivate packet. The local site inactivates the project
and all accounts on the project and sends a reply to the XDCDB in a notify_project_inactivate
packet. The XDCDB finishes the transaction by sending an inform_transaction_complete.
The sequence of packets is:
[width=50%]
[cols="2,1"]
|===
| `request_project_inactivate` | incoming
| `notify_project_inactivate` | outgoing
| `inform_transaction_complete` | incoming
|===
===== `request_project_inactivate` Packet
The `request_project_inactivate` packet contains information to be used by a local site to identify
the project to be inactivated. The result of this request should be the inactivation of the project as
well as inactivation of all accounts on the project.
The contents of the packet are as follows, with required tags in *bold*.
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *ProjectID* | | | afm
| *ResourceList* | | 0 | xx.ncsa.xsede
| `Comment` | | | Project out of funds
| `AllocatedResource` | | | staff.xsede
| `GrantNumber` | | | AST040009
| `StartDate` | | | 2006-04-01
| `EndDate` | | | 2007-03-31
| `ServiceUnitsAllocated` | | | 50000
| `ServiceUnitsRemaning` | | | -2345
|===
===== `notify_project_inactivate` Packet
After having inactivated the project and all related accounts, the local site must reply with a
`notify_project_inactivate` packet. This packet must contain the same project information as the
incoming `request_project_inactivate`.
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *ProjectID* | | | afm
| *ResourceList* | | 0 | xx.ncsa.xsede
|===
===== `request_project_inactivate` Section of `process_packets.py`
-----
if packet_type == 'request_project_inactivate':
resource = packet.ResourceList[0]
project_id = packet.ProjectID
# SP: inactivate the project and all accounts on the project
npi = packet.reply_packet()
amie_client.send_packet(npi)
-----
===== `inform_transaction_complete` Packet
The local site will then receive an `inform_transaction_complete` to complete the transaction.
==== `request_project_reactivate` Transaction
The purpose of this transaction is to ask a local site to reactivate a project that was previously
inactivated. The XDCDB sends the project information in a `request_project_reactivate` packet.
The local site reactivates the project and the account for the PI and sends a reply to the XDCDB in
a `notify_project_reactivate` packet. The XDCDB finishes the transaction by sending an
`inform_transaction_complete`.
The sequence of packets is
[width=50%]
[cols="2,1"]
|===
| `request_project_reactivate` | incoming
| `notify_project_reactivate` | outgoing
| `inform_transaction_complete` | incoming
|===
===== `request_project_reactivate` Packet
The `request_project_reactivate` packet contains information to be used by a local site to reactivate
a project and the account for the PI on the project. No other accounts should be reactivated.
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| `PersonID` | | | 21619
| *ProjectID* | | | afm
| `Comment` | | | PI begged for more time.
| *ResourceList* | | 0 | hc.ncsa.xsede
| `AllocatedResource` | | | staff.xsede
| `GrantNumber` | | | AST040009
| `StartDate` | | | 2006-04-01
| `EndDate` | | | 2007-03-31
| `ServiceUnitsAllocated` | | | 50000
| `ServiceUnitsRemaining` | | | 12345
|===
===== `notify_project_reactivate` Packet
After having reactivated a project, the local site must reply with a `notify_project_reactivate` packet.
This packet must contain the same project information as the incoming `request_project_reactivate`.
An inactive project can also be reactivated with a `request_project_create` packet.
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *ProjectID* | | | afm
| *ResourceList* | | 0 | hc.ncsa.xsede
|===
===== `request_project_reactivate` Section of `process_packets.py`
-----
if packet_type == 'request_project_reactivate':
resource = packet.ResourceList[0]
project_id = packet.ProjectID
pi_person_id = packet.PersonID
# SP: reactivate the project and the PI account on the project (but no other accounts)
npr = packet.reply_packet()
amie_client.send_packet(npr)
-----
===== `inform_transaction_complete` Packet
==== `request_account_inactivate` Transaction
The purpose of this transaction is to ask a local site to inactivate an account for a user on a project.
The XDCDB sends the account information in a `request_account_inactivate` packet. The local site
inactivates the account and sends a reply to the XDCDB in a `notify_account_inactivate packet`.
The XDCDB finishes the transaction by sending an `inform_transaction_complete`.
The sequence of packets is:
[width=50%]
[cols="2,1"]
|===
| `request_account_inactivate` | incoming
| `notify_account_inactivate` | outgoing
| `inform_transaction_complete` | incoming
|===
===== `request_account_inactivate` Packet
The `request_account_inactivate` packet contains information to be used by a local site to inactivate
an account on a project.
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *PersonID* | | | 21619
| *ProjectID* | | | afm
| `Comment` | | | No longer on the project.
| *ResourceList* | | 0 | hc.ncsa.xsede
|===
===== `notify_account_inactivate` Packet
After having inactivated an account, the local site must reply with a `notify_account_inactivate`
packet. This packet must contain the same account information as the incoming
`request_account_inactivate`.
*NOTE*: it is important that the site inactivate the account, rather than remove it. It is possible that
the account could be reactivated in the future. Also, account inactivation does not imply user
inactivation. The *PersonID* (as well as the related login on the resource) must be retained for this
user, since the XDCDB will retain them as well.
[width=75%]
[cols="3,2,1,9"]
|===
| tag | subtag | seq | value
| *PersonID* | | | 21619
| *ProjectID* | | | afm
| *ResourceList* | | 0 | hc.ncsa.xsede
|===
===== `request_account_inactivate` Section of `process_packets.py`
-----
if packet_type == 'request_account_inactivate':
resource = packet.ResourceList[0]
project_id = packet.ProjectID
person_id = packet.PersonID
# SP: inactivate the account on the project
nai = packet.reply_packet()
amie_client.send_packet(nai)
-----
===== `inform_transaction_complete` Packet
==== `request_account_reactivate` Transaction
The purpose of this transaction is to ask a local site to reactivate an account for a user on a project
that was previously inactivated. The XDCDB sends the account information in a
`request_account_reactivate` packet. The local site reactivates the account and sends a reply to the
XDCDB in a `notify_account_reactivate` packet. The XDCDB finishes the transaction by sending
an `inform_transaction_complete`.
*NOTE*: this transaction is included in this documentation for completeness only. The XDCDB
currently will reactivate accounts using the `request_account_create` transaction.
The sequence of packets is
[width=50%]
[cols="2,1"]
|===
| `request_account_reactivate` | incoming
| `notify_account_reactivate` | outgoing
| `inform_transaction_complete` | incoming
|===
===== `request_account_reactivate` Packet
The `request_account_reactivate` packet contains information to be used by a local site to reactivate
an account on a project.