|
1 ## ICONOLAB-MCC |
|
2 |
|
3 ### 1. Configuration and setup for development |
|
4 |
|
5 ### virtualenv |
|
6 |
|
7 - Install pip |
|
8 - Create a virtualenv for the project (using virtualenvwrapper is a good idea if possible). Python version is 3.6.5 |
|
9 |
|
10 Example |
|
11 ``` |
|
12 virtualenv -p python3.6 ./virtualenv |
|
13 . virtualenv/bin/activate |
|
14 |
|
15 cd src/ |
|
16 pip install -r requirements/dev.txt |
|
17 ``` |
|
18 |
|
19 |
|
20 ### Django project setup |
|
21 |
|
22 - Install iconolab |
|
23 pip install -e path/iconolab |
|
24 |
|
25 - Copy iconolab/src/settings/dev.py.tmpl into iconolab-mcc/src/iconolab_mcc/settings/dev.py, adapt content to configuration |
|
26 - cd into iconolab-mcc/src folder and run |
|
27 |
|
28 python manage.py migrate |
|
29 |
|
30 to create database tables |
|
31 |
|
32 - Run |
|
33 |
|
34 python manage.py createsuperuser |
|
35 |
|
36 to create an admin user |
|
37 |
|
38 - Collect static files |
|
39 |
|
40 python manage.py collectstatic |
|
41 |
|
42 |
|
43 - Use Docker adress for HOST in settings |
|
44 |
|
45 - don't use os.path.join(BASE_DIR, 'media') in settings |
|
46 |
|
47 - make JS_DEV_MODE afalse (if not, it will not connect correctly on server) |
|
48 |
|
49 |
|
50 |
|
51 ### Iconolab to Iconolab-mcc |
|
52 |
|
53 - Create uploads and cache repertories |
|
54 MEDIA_ROOT/uploads and MEDIA_ROOT/cache |
|
55 |
|
56 - Move the run directory into iconolab-mcc |
|
57 |
|
58 - Create static directory |
|
59 |
|
60 - Edit the settings in dev.py (using absolute path is better) |
|
61 BASE_DIR = '../iconolab-mcc/src/iconolab_mcc' |
|
62 |
|
63 STATIC_ROOT = '../iconolab-mcc/run/web/static/site' |
|
64 MEDIA_ROOT = '../iconolab-mcc/run/web/media' |
|
65 |
|
66 - Create templates and views directories |
|
67 |
|
68 iconolab-mcc/src/iconolab_mcc/templates |
|
69 iconolab-mcc/src/iconolab_mcc/views |
|
70 |
|
71 |
|
72 - Add the charter, credits and legalmentions templates and it views and URLs (the URLs need to be on top) |
|
73 |
|
74 exemple for credits view in iconolab-mcc/src/iconolab_mcc/views/misc.py : |
|
75 |
|
76 class CreditsView(TemplateView): |
|
77 template_name='iconolab_mcc/misc/credits.html' |
|
78 |
|
79 exemple for credits url in iconolab-mcc/src/urls.py : |
|
80 |
|
81 path('credits/', views.misc.CreditsView.as_view(), name="iconolab_mcc_credits"), |
|
82 |
|
83 |
|
84 - Add line "from iconolab_mcc import" views in file iconolab_mcc/urls.py to access views |
|
85 |
|
86 |
|
87 - Change "from iconolab.settings import *" into "from iconolab_mcc.settings import *" in the settings |
|
88 |
|
89 |
|
90 - Add help and footer templates |
|
91 |
|
92 iconolab-mcc/src/iconolab_mcc/templates/misc/help.html |
|
93 iconolab-mcc/src/iconolab_mcc/templates/partials/footer.html |
|
94 |
|
95 |
|
96 |
|
97 |
|
98 - Add logo_mcc_footer in src/iconolab_mcc/static/iconolab_mcc/img |
|
99 |
|
100 |
|
101 |
|
102 ### Docker |
|
103 |
|
104 Migrate the file docker-compose.yml in iconolab-mcc project |
|
105 |
|
106 - Create containers |
|
107 |
|
108 docker-compose up |
|
109 |
|
110 - Run containers |
|
111 |
|
112 docker-compose start |
|
113 |
|
114 |
|
115 ### Elasticsearch |
|
116 |
|
117 Some objects in Iconolab are indexed and searched using ElasticSearch. You need to configure Haystack (see dev.py.tmpl, HAYSTACK_CONNECTIONS) and run: |
|
118 |
|
119 python manage.py rebuild_index |
|
120 |
|
121 |
|
122 |
|
123 ### 2. Python server |
|
124 |
|
125 - cd into the iconolab/src folder and run |
|
126 |
|
127 python manage.py runserver |
|
128 |
|
129 By default, the app is accessible through http://127.0.0.1:8000/home |
|
130 |
|
131 |
|
132 ### 3. Importing initial data from CSV |
|
133 |
|
134 Make sure to have the following in the same folder: |
|
135 |
|
136 * All the images to import. The image names must match their respective item inventory number. |
|
137 * A csv file that contains the metadata for the items you will import |
|
138 * A json file for initializing the collection in the database. (Optional if you want to import images in an existing collection) |
|
139 * A json file for the metacategories that will be linked to the collection. |
|
140 * Ensure the folder settings.MEDIA_ROOT+/uploads/ exists |
|
141 |
|
142 The following django manage.py command is used to import collection data and images: |
|
143 |
|
144 ``` |
|
145 python manage.py importimages <:export-csv-path> --delimiter <:delimiter> --encoding <:encoding> --collection-json <:collection_fixture_FILENAME> (OR --collection-id <:collection_id> if collection already exists in db) --metacategories-json <:metacategories_json_FILENAME> |
|
146 ``` |
|
147 |
|
148 Options: |
|
149 - ```--delimiter```: the delimiter for the csv file. For special ascii characters add a # before the code. Supported special chars are 9 (tab), 29 (Group separator), 30 (Record separator), 31 (Unit separator) |
|
150 - ```--encoding```: the encoding provided if the csv is not in utf-8. Exemple: 8859 for ISO-8859 |
|
151 - ```--collection-json```: the json file to create the collection from |
|
152 - ```--collection-id```: the id of the collection to import into, it must already exist |
|
153 - ```--metacategories-json```: the json file to create metacategories on the collection we're importing into |
|
154 - ```--jpeg-quality```: the jpeg quality: default to the setting IMG_JPG_DEFAULT_QUALITY |
|
155 - ```--no-jpg-conversion```: set to True so the command will not convert the images to jpg. Useful for pre-converted jpeg and especially when importing large image banks |
|
156 - ```--img-filename-identifier```: the column from which the command will try to find images in the folder: use keys from the setting IMPORT_FIELDS_DICT. Default is "INV". |
|
157 - ```--filename-regexp-prefix```: allows you to customize the way the command try to find images by specifying a regexp pattern to match *before* the identifier provided in img-filename-identifier. Defaults to .* |
|
158 - ```--filename-regexp-suffix```: allows you to customize the way the command try to find images by specifying a regexp pattern to match *after* the identifier provided in img-filename-identifier. Defaults to [\.\-_].* |
|
159 |
|
160 Notes: |
|
161 * The export csv path will be used to find everything else (images and fixtures files). |
|
162 * If the csv file is not encoded in utf-8, you MUST provide --encoding so the csv file can be read |
|
163 * You MUST provide either --collection-json or --collection-id, else the command doesn't know to which collection the objects will belong to. |
|
164 * To find all images for a given item, the command will try to match filenames according to the pattern build from the 3 options: filename-regexp-prefix+<value of img-filename-identifier>+filename-regexp-suffix. For instance by default, for an object with an INV of MIC.3.10, the files MIC.3.10.jpg and MIC.3.10.verso.jpg would be matched and linked to the object. |
|
165 * The command will first parse the csv, then create the objects in the database (Item and ItemMetadata), then move the images to the settings.MEDIA_ROOT+/uploads/ folder after converting them to JPEG, then create the database objects for the images. The command will ignore any csv row that lacks an image or any csv row that already has a database entry for the collection (by default INV number is used to test if a database entry exists). |
|
166 |
|
167 |
|
168 ### 4. Updating already existing data |
|
169 |
|
170 Another management command allows for editing data using only a .csv file. The command will go through the csv and update the metadatas for every objects it finds in the database with the csv row content. |
|
171 |
|
172 ``` |
|
173 python manage.py updatecollection --collection-id=<:id> --delimiter=<:delimiter> --encoding=<:encoding> |
|
174 ``` |
|
175 |
|
176 Options: |
|
177 - ```--delimiter```: the delimiter for the csv file. For special ascii characters add a # before the code. Supported special chars are 9 (tab), 29 (Group separator), 30 (Record separator), 31 (Unit separator) |
|
178 - ```--encoding```: the encoding provided if the csv is not in utf-8. Exemple: 8859 for ISO-8859 |
|
179 - ```--collection-id```: the id of the collection to import into, it must already exist |
|
180 |
|
181 |
|
182 * document the test import command : `python manage.py importimages --collection-json dossierImportMontauban/montauban_collection.json --metacategories-json dossierImportMontauban/montauban_metacategories_import.json --encoding "UTF-8" --delimiter "," dossierImportMontauban/ExportMontauban.csv` |