Difference between revisions of "CKAN Administration"

From WPRDC Wiki
Jump to navigation Jump to search
(Add bit about using CKAN metadata for centralizing operational ETL data)
Line 31: Line 31:
To add or change the departments belonging to a particular publisher organization edit the <code>dataset_schema.json</code> file: <code>> vi /usr/lib/ckan/default/src/ckanext-scheming/ckanext/scheming/dataset_schema.json</code>
To add or change the departments belonging to a particular publisher organization edit the <code>dataset_schema.json</code> file: <code>> vi /usr/lib/ckan/default/src/ckanext-scheming/ckanext/scheming/dataset_schema.json</code>


Then run <code>>sudo service apache2 reload</code>
Then run <code>> sudo service apache2 reload</code>


The extra tricky part about this one is that [https://github.com/WPRDC/ckanext-wprdctheme our GitHub repository that includes this JSON file] is installed in a different directory: <code>/usr/lib/ckan/default/src/ckanext-wprdctheme/</code> but changes to the files in that directory (and subdirectories) do nothing.
The extra tricky part about this one is that [https://github.com/WPRDC/ckanext-wprdctheme our GitHub repository that includes this JSON file] is installed in a different directory: <code>/usr/lib/ckan/default/src/ckanext-wprdctheme/</code> but changes to the files in that directory (and subdirectories) do nothing.


== Other changes ==
== Other changes ==
=== Using CKAN metadata instead of local caches ===
To avoid keeping local databases about datasets (for instance, when writing code to track some aspect of datasets), store such information (such as the last time an ETL job was run on a given package) in the 'extras' metadata field of the CKAN package, as much as possible. This stores information in a centralized location so ETL jobs can be run from multiple computers without any other coordination. The extras metadata fields are cataloged on the [[CKAN Metadata]] page.
=== Hacky workaround for adding new users to publishers ===
=== Hacky workaround for adding new users to publishers ===
In additional to adding the users to the organizations through the CKAN front-end, you also have to add them to groups, using this URL: [http://wprdc.org/group-adder/]
In additional to adding the users to the organizations through the CKAN front-end, you also have to add them to groups, using this URL: [http://wprdc.org/group-adder/]
== Other CKAN stuff ==
See the CKAN section of [https://github.com/WPRDC/knowhow-and-tricks].


[[Category:Onboarding]]
[[Category:Onboarding]]

Revision as of 14:06, 1 March 2022

Changes that can be made through the frontend

There's a lot of documentation on publishing data on our CKAN portal here.

A few samples (to eventually migrate over):

Changes that can be made through the backend

Configuring the CKAN server

(The contents of this section were initially taken from the ORIENTATION file in /home/ubuntu on the CKAN production server.)

  • The main CKAN config file is at /etc/ckan/default/production.ini
  • To monitor HTTP requests in real-time: > tail -f /var/log/nginx/access.log
  • Service-worker activity (like the Express Loader uploading files to the datastore and background geocoding) can be found in: /var/log/ckan-worker.log
  • Edit templates here (changes to templates should show up when reloading the relevant web pages): /usr/lib/ckan/default/src/ckanext-wprdctheme/ckanext/wprdc/templates
  • templates/terms.html is the source for the pop-up version of the Terms of Use. There appears to be no template linked to the "Terms" hyperlink.
  • Create a file templates/foo.html and then run > sudo service supervisor restart and THEN load data.wprdc.org/foo.html in your browser, and the page will be there.
  • Presumably data.wprdc.org/foo/ can be populated by creating a file at templates/foo/index.html.

Managing the CKAN server

  • To restart the Express Loader: > sudo supervisorctl restart ckan-worker:*

Adding/changing departments of publishers

To add or change the departments belonging to a particular publisher organization edit the dataset_schema.json file: > vi /usr/lib/ckan/default/src/ckanext-scheming/ckanext/scheming/dataset_schema.json

Then run > sudo service apache2 reload

The extra tricky part about this one is that our GitHub repository that includes this JSON file is installed in a different directory: /usr/lib/ckan/default/src/ckanext-wprdctheme/ but changes to the files in that directory (and subdirectories) do nothing.

Other changes

Using CKAN metadata instead of local caches

To avoid keeping local databases about datasets (for instance, when writing code to track some aspect of datasets), store such information (such as the last time an ETL job was run on a given package) in the 'extras' metadata field of the CKAN package, as much as possible. This stores information in a centralized location so ETL jobs can be run from multiple computers without any other coordination. The extras metadata fields are cataloged on the CKAN Metadata page.

Hacky workaround for adding new users to publishers

In additional to adding the users to the organizations through the CKAN front-end, you also have to add them to groups, using this URL: [1]