src/README.md
author durandn
Mon, 05 Dec 2016 16:51:30 +0100
changeset 261 0d89d1066874
parent 154 79b70254a5e0
child 287 959cbaad2076
permissions -rw-r--r--
updated Readmes
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     1
# How to start?
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     2
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     3
1. Make sure PIP is installed then install Django and others dependencies with 
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     4
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     5
```
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     6
pip install -r requirements.txt
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     7
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     8
```
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
     9
44
6730ec4d7e37 fixing js dependencies
Harris Baptiste <harris.baptiste@iri.centrepompidou.fr>
parents: 33
diff changeset
    10
2. Move to src/iconolab/static/js/iconolab-bundle to install js dependencies.
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    11
Make sure your have installed nodejs then run the command bellow
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    12
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    13
```
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    14
npm install
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    15
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    16
```
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    17
3. To recreate the bundle file that lives in dist/
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    18
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    19
```
46
86608f92ebed fixing tags
Harris Baptiste <harris.baptiste@iri.centrepompidou.fr>
parents: 44
diff changeset
    20
npm run build
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    21
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    22
```
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    23
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    24
4. To add a new js module, you can add it to the js/components folder and then run
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    26
```
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    27
npm start
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    28
```
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    29
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    30
## ICONOLAB ##
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    31
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    32
### 1. Configuration and setup
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    33
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    34
#### virtualenv
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    35
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    36
- Install pip
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    37
- Create a virtualenv for the project (using virtualenvwrapper is a good idea if possible). Python version is 3.5.1
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    38
- Run
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    39
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    40
	pip install -r requirements.txt
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    41
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    42
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    43
#### node.js
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    44
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    45
- Make sure nodejs is installed
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    46
- cd into iconolab/src/iconolab/static/iconolab/js and run
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    47
	
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    48
	npm install
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    49
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    50
- To recreate the bundle file that lives in dist/
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    51
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    52
	npm build
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    53
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    54
- To add a new js module, you can add it to the js/components folder and then run
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    55
	
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    56
	npm start
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    57
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    58
#### Django project setup
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    59
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    60
- Copy iconolab/src/settings/dev.py.tmpl into iconolab/src/settings/dev.py, adapt content to configuration
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    61
- cd into iconolab/src folder and run
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    62
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    63
    python manage.py migrate
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    64
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    65
to create database tables
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    66
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    67
- Run
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    68
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    69
    python manage.py createsuperuser
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    70
    
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    71
to create an admin user
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    72
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    73
#### Elasticsearch
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    74
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    75
Some objects in Iconolab are indexed and searched using ElasticSearch. You need to configure Haystack (see dev.py.tmpl, HAYSTACK_CONNECTIONS) and run:
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    76
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    77
	python manage.py rebuild_index
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    78
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    79
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    80
### 2. Development server
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    81
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    82
#### 2.1 Python server
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    83
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    84
- cd into the iconolab/src folder and run
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    85
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    86
	python manage.py runserver
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    87
	
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    88
By default, the app is accessible through http://127.0.0.1:8000/home
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
    89
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    90
#### 2.2 Javascript development
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    91
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    92
- cd into the iconolab/src_js/iconolab-bundle folder and run
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    93
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    94
	npm install
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    95
	npm run start
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    96
	
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    97
This will serve the iconolab.js file in the iconolab/src/iconolab/static/js and update it on changes you make in the js code in src_js so you can
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    98
edit the code and debug it live in your browser
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
    99
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   100
### 3. Importing initial data from CSV
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
   101
140
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   102
Make sure to have the following in the same folder:
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
   103
140
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   104
* All the images to import. The image names must match their respective item inventory number.
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   105
* A csv file that contains the metadata for the items you will import
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   106
* A json file for initializing the collection in the database. (Optional if you want to import images in an existing collection)
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   107
* A json file for the metacategories that will be linked to the collection.
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   108
* Ensure the folder settings.MEDIA_ROOT+/uploads/ exists
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
   109
140
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   110
The following django manage.py command is used to import collection data and images:
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
   111
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   112
	python manage.py importimages <:export-csv-path> --delimiter <:delimiter> --encoding <:encoding> --collection-json <:collection_fixture_FILENAME> (OR --collection-id <:collection_id> if collection already exists in db) --metacategories-json <:metacategories_json_FILENAME> 
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   113
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   114
Options:
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   115
 --delimiter: the delimiter for the csv file. For special ascii characters add a # before the code. Supported special chars are 9 (tab), 29 (Group separator), 30 (Record separator), 31 (Unit separator)
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   116
 --encoding: the encoding provided if the csv is not in utf-8. Exemple: 8859 for ISO-8859
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   117
 --collection-json: the json file to create the collection from
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   118
 --collection-id: the id of the collection to import into, it must already exist
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   119
 --metacategories-json: the json file to create metacategories on the collection we're importing into
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   120
 --jpeg-quality: the jpeg quality: default to the setting IMG_JPG_DEFAULT_QUALITY
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   121
 --no-jpg-conversion: set to True so the command will not convert the images to jpg. Useful for pre-converted jpeg and especially when importing large image banks
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   122
 --img-filename-identifier: the column from which the command will try to find images in the folder: use keys from the setting IMPORT_FIELDS_DICT. Default is "INV".
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   123
 --filename-regexp-prefix: allows you to customize the way the command try to find images by specifying a regexp pattern to match *before* the identifier provided in img-filename-identifier. Defaults to .*
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   124
 --filename-regexp-suffix: allows you to customize the way the command try to find images by specifying a regexp pattern to match *after* the identifier provided in img-filename-identifier. Defaults to [\.\-_].*
25
843863132ac9 Readme edit and initial data fixture
durandn
parents: 15
diff changeset
   125
	
140
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   126
Notes: 
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   127
* The export csv path will be used to find everything else (images and fixtures files). 
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   128
* If the csv file is not encoded in utf-8, you MUST provide --encoding so the csv file can be read
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   129
* You MUST provide either --collection-json or --collection-id, else the command doesn't know to which collection the objects will belong to.
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   130
* To find all images for a given item, the command will try to match filenames according to the pattern build from the 3 options: filename-regexp-prefix+<value of img-filename-identifier>+filename-regexp-suffix. For instance by default, for an object with an INV of MIC.3.10, the files MIC.3.10.jpg and MIC.3.10.verso.jpg would be matched and linked to the object.
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   131
* The command will first parse the csv, then create the objects in the database (Item and ItemMetadata), then move the images to the settings.MEDIA_ROOT+/uploads/ folder after converting them to JPEG, then create the database objects for the images. The command will ignore any csv row that lacks an image or any csv row that already has a database entry for the collection (by default INV number is used to test if a database entry exists).
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   132
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   133
### 4. Updating already existing data
140
b0aae3bc25d2 updated readme for importimages command
durandn
parents: 46
diff changeset
   134
261
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   135
Another management command allows for editing data using only a .csv file. The command will go through the csv and update the metadatas for every objects it finds in the database with the csv row content.
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   136
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   137
	python manage.py updatecollection --collection-id=<:id> --delimiter=<:delimiter> --encoding=<:encoding>
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   138
	
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   139
Options:
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   140
 --delimiter: the delimiter for the csv file. For special ascii characters add a # before the code. Supported special chars are 9 (tab), 29 (Group separator), 30 (Record separator), 31 (Unit separator)
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   141
 --encoding: the encoding provided if the csv is not in utf-8. Exemple: 8859 for ISO-8859
0d89d1066874 updated Readmes
durandn
parents: 154
diff changeset
   142
 --collection-id: the id of the collection to import into, it must already exist