# HG changeset patch # User Nicolas DURAND # Date 1425303562 -3600 # Node ID ddba4624d66117ba8a01fac49499a249f0065f1a # Parent 5bd3fb0233965e5b81d2d37b376ad5659d3a87ce Added cache support for getting issues, commits and comments + reworked workshop page, now displaying latest changes and latest opened issues and links to changeset and issues list + changeset page now displays a change summary diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/models.py --- a/src/catedit/models.py Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/models.py Mon Mar 02 14:39:22 2015 +0100 @@ -6,6 +6,7 @@ """ from rdflib import Graph, RDF, RDFS, Literal, URIRef +from rdflib.compare import to_isomorphic, graph_diff from uuid import uuid4 from io import StringIO from slugify import slugify diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/static/css/style.css --- a/src/catedit/static/css/style.css Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/static/css/style.css Mon Mar 02 14:39:22 2015 +0100 @@ -20,10 +20,15 @@ .navbar-form { width: 300px; - vertical-align: center; + vertical-align: middle; display: inline-block; } +.cat-changes-item +{ + vertical-align: middle; +} + .select-repo { float:right; diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/static/js/property_functions.js --- a/src/catedit/static/js/property_functions.js Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/static/js/property_functions.js Mon Mar 02 14:39:22 2015 +0100 @@ -82,7 +82,7 @@ displayCorrespondingField : function() { propertySelectItem = document.getElementById("property_selector"); - switch(propertySelectItem[propertySelectItem.selectedIndex].label) + switch(propertySelectItem[propertySelectItem.selectedIndex].className) { case "property_type_default": document.getElementById("literal-field").className = "hidden form-control"; diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/categories/editor.html --- a/src/catedit/templates/categories/editor.html Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/templates/categories/editor.html Mon Mar 02 14:39:22 2015 +0100 @@ -64,7 +64,7 @@ Liste des propriétés ... {% for predicate in config["PROPERTY_LIST"] %} - + {% endfor %} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/categories/workshop.html --- a/src/catedit/templates/categories/workshop.html Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/templates/categories/workshop.html Mon Mar 02 14:39:22 2015 +0100 @@ -42,7 +42,6 @@ {% endblock repo_list %} {% block page_content %}

CatEdit - {{current_repository}}

-

Créer une catégorie :

{% if session["user_logged"] and not session["user_can_edit"] %} +

Créer une catégorie :

{% if session.get("user_logged") %}
@@ -97,5 +97,77 @@

Soumettre mes changements actuels :

{% endif %} -

Discussions sur cet ensemble de catégories :

+

Modifications récentes

+ + + + + + + + + + + {% if not changeset_list %} + + + + {% else %} + {% for changeset in changeset_list %} + + + + + + + + {% endfor %} + {% endif %} + +
AuteurDateMessage de soumissionCommentaires
+ Il n'y a aucune modification à afficher. +
{{changeset["author"]}}{{changeset["date"]}}{{changeset["title"]}}{{changeset["comment_count"]}} + + +
+

Voir la liste des modifications: +

+

Discussions récentes

+ + + + + + + + + + + + {% if not discussion_list %} + + + + {% else %} + {% for discussion in discussion_list %} + + + + + + + + + {% endfor %} + {% endif %} + +
AuteurDateTitreCommentairesDernier commentaire
+ Il n'y a de discussion à afficher pour cette ensemble de catégories. Créer une discussion +
{{discussion["author"]}}{{discussion["opening_date"]}}{{discussion["title"]}}{{discussion["comment_count"]}}{{discussion["last_updated"]}} + + +
+

Voir la liste des discussions: {% endblock page_content %} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/macros.html --- a/src/catedit/templates/macros.html Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/templates/macros.html Mon Mar 02 14:39:22 2015 +0100 @@ -1,12 +1,15 @@ -{% macro category_table(cat_list, current_repository, state_list=["mixed"], target="", interactive=True) -%} +{% macro category_table(cat_list, current_repository, state_list=["mixed"], target="", interactive=True, with_colors=True) -%} {% for cat in cat_list %} {% if state_list == ["mixed"] or (cat.state in state_list) %} - {{ cat.cat_label }} {{ cat.cat_description}} @@ -48,12 +51,14 @@ {% endif %}
diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/social/changeset.html --- a/src/catedit/templates/social/changeset.html Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/templates/social/changeset.html Mon Mar 02 14:39:22 2015 +0100 @@ -23,13 +23,13 @@ $(".changes-table-toggle").removeClass("btn-info") $(".changes-box").removeClass("alert-info"); - $(".before-cat-table").slideToggle(function(){ + $(".before-cat-table").toggle("fast", function(){ $(".before-cat-table-toggle").children().toggleClass("glyphicon-chevron-up glyphicon-chevron-down") }); - $(".after-cat-table").slideUp(function(){ + $(".after-cat-table").hide("fast", function(){ $(".after-cat-table-toggle").children().removeClass("glyphicon-chevron-up").addClass("glyphicon-chevron-down"); }); - $(".changes-table").slideUp(function(){ + $(".changes-table").hide(function(){ $(".changes-table-toggle").children().removeClass("glyphicon-chevron-up").addClass("glyphicon-chevron-down"); }); }); @@ -44,13 +44,13 @@ $(".changes-table-toggle").removeClass("btn-info") $(".changes-box").removeClass("alert-info"); - $(".after-cat-table").slideToggle(function(){ + $(".after-cat-table").toggle("fast", function(){ $(".after-cat-table-toggle").children().toggleClass("glyphicon-chevron-up glyphicon-chevron-down") }); - $(".before-cat-table").slideUp(function(){ + $(".before-cat-table").hide("fast", function(){ $(".before-cat-table-toggle").children().removeClass("glyphicon-chevron-up").addClass("glyphicon-chevron-down"); }); - $(".changes-table").slideUp(function(){ + $(".changes-table").hide(function(){ $(".changes-table-toggle").children().removeClass("glyphicon-chevron-up").addClass("glyphicon-chevron-down"); }); }); @@ -68,10 +68,10 @@ $(".changes-table").slideToggle(function(){ $(".changes-table-toggle").children().toggleClass("glyphicon-chevron-up glyphicon-chevron-down") }); - $(".before-cat-table").slideUp(function(){ + $(".before-cat-table").hide("fast", function(){ $(".before-cat-table-toggle").children().removeClass("glyphicon-chevron-up").addClass("glyphicon-chevron-down"); }); - $(".after-cat-table").slideUp(function(){ + $(".after-cat-table").hide("fast", function(){ $(".after-cat-table-toggle").children().removeClass("glyphicon-chevron-up").addClass("glyphicon-chevron-down"); }); }); @@ -155,13 +155,46 @@
- - - - -
- placeholder -
+
    +
  • +

    + Catégories ajoutées + {{differences["additions"]|length}} +

    +
  • + {% for added_category in differences["additions"] %} +
  • + {{ added_category }} +
  • + {% endfor %} +
  • +

    + Catégories modifiées + {{differences["modifications"]|length}} +

    +
  • + {% for modified_category in differences["modifications"] %} +
  • + {{ modified_category[0] }} + + {{ modified_category[1] }} + + + +
  • + {% endfor %} +
  • +

    + Catégories supprimées + {{differences["deletions"]|length}} +

    +
  • + {% for deleted_category in differences["deletions"] %} +
  • + {{ deleted_category }} +
  • + {% endfor %} +
@@ -187,13 +220,14 @@ {% else %} {% import "macros.html" as macros %} - {{ macros.category_table(new_cat_list, current_repository, state_list=["modified"], interactive=False) }} + {{ macros.category_table(new_cat_list, current_repository, state_list=["modified"], interactive=False, with_colors=False) }} {% endif %} {% endif %}
{% endblock additional_content %} +{% block back_link %}{{ url_for('social.changesets_index', repository=current_repository)}}{% endblock back_link %} {% block comment_posting_target %}{{url_for("social.changeset", changeset_id=changeset_id, repository=current_repository)}}{% endblock %} {% block page_content %} {{super()}} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/social/changesets_index.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/catedit/templates/social/changesets_index.html Mon Mar 02 14:39:22 2015 +0100 @@ -0,0 +1,72 @@ +{% extends "layout.html" %} +{% if not session["user_logged"] or not session["user_can_edit"][current_repository] %} + {% set readonly="readonly" %} +{% else %} + {% set readonly=False %} +{% endif %} +{% block title %} {{ current_repository}}: Espaces de discussion {% endblock title %} +{% block head %} + {{ super() }} +{% endblock head %} +{% block navbar_items %} + {{ super() }} + {% if session.get("user_logged", None) %} +
  • >
  • +
  • Atelier
  • +
  • >
  • +
  • Social
  • + {% endif %} +{% endblock navbar_items %} +{% block repo_list %} + {{ super() }} +{% endblock repo_list %} +{% block page_content %} +

    CatEdit - {{current_repository}}

    + {% if readonly %} + + {% endif %} +

    Historique des modifications

    + + + + + + + + + + + {% if not changeset_list %} + + + + {% else %} + {% for changeset in changeset_list %} + + + + + + + + {% endfor %} + {% endif %} + + + + +
    AuteurDateMessage de soumissionCommentaires
    + Il n'y a aucune modification à afficher. +
    {{changeset["author"]}}{{changeset["date"]}}{{changeset["title"]}}{{changeset["comment_count"]}} + + +
    + {% import "macros.html" as macros %} + {{ macros.render_pagination(commits_pagination, commits_per_page, page_arg_name="commits_page", per_page_arg_name="commits_per_page") }} +
    +{% endblock page_content%} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/social/comment_thread_layout.html --- a/src/catedit/templates/social/comment_thread_layout.html Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/templates/social/comment_thread_layout.html Mon Mar 02 14:39:22 2015 +0100 @@ -8,8 +8,6 @@ {% if session.get("user_logged", None) %}
  • >
  • Atelier
  • -
  • >
  • -
  • Social
  • {% endif %} {% endblock navbar_items %} {% block page_content %} @@ -17,66 +15,68 @@
    {% block additional_content %} {% endblock additional_content %} -

    Discussion

    - {% if comment_form.comment_field.errors %} -
    - - - Erreur: - - Votre commentaire est vide. -
    - {% endif %} - - - - - - +
    +

    Discussion

    + {% if comment_form.comment_field.errors %} +
    + + + Erreur: + + Votre commentaire est vide. +
    + {% endif %} +
    {{comments["author"]}} - {{comments["opening_date"]}} - Titre: {{comments["title"]}} -
    - {{comments["opening_post"]}}
    + + + - - - {% if comments["comment_list"]|length == 0 %} - - - - {% else %} - {% for comment in comments["comment_list"] %} + + + + + + {% if comments["comment_list"]|length == 0 %} - - - + - {% endfor %} - {% endif %} - - - - - - - - -
    {{comments["author"]}} + {{comments["opening_date"]}} + Titre: {{comments["title"]}}
    Aucun commentaire à afficher
    + {{comments["opening_post"]}}
    {{comment["author"]}} {{comment["date"]}} {{comment["body"]}} Aucun commentaire à afficher
    - {% import "macros.html" as macros %} - {{ macros.render_pagination(comments_pagination, comments_per_page) }} -
    - {{ comment_form.comment_field.label }} - - -
    - {{ comment_form.hidden_tag() }} -
    - {{ comment_form.comment_field(class="form-control", readonly=readonly) }} -
    - - Retour -
    - -
    + {% else %} + {% for comment in comments["comment_list"] %} + + {{comment["author"]}} + {{comment["date"]}} + {{comment["body"]}} + + {% endfor %} + {% endif %} + + + {% import "macros.html" as macros %} + {{ macros.render_pagination(comments_pagination, comments_per_page) }} + + + + + {{ comment_form.comment_field.label }} + + +
    +
    + {{ comment_form.hidden_tag() }} +
    + {{ comment_form.comment_field(class="form-control", readonly=readonly) }} +
    + + Retour +
    +
    + + + + +
    {% endblock page_content %} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/social/discussion.html --- a/src/catedit/templates/social/discussion.html Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/templates/social/discussion.html Mon Mar 02 14:39:22 2015 +0100 @@ -19,7 +19,7 @@ {% endfor %} {% endblock comment_thread_options %} - +{% block back_link %}{{ url_for('social.discussions_index', repository=current_repository)}}{% endblock back_link %} {% block page_content %} {% if discussion_id == "new" %}

    CatEdit - {{current_repository}}

    @@ -45,7 +45,7 @@ {{ comment_form.comment_field(class="form-control", readonly=readonly) }} - Retour + Retour {% else %} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/social/discussions_index.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/catedit/templates/social/discussions_index.html Mon Mar 02 14:39:22 2015 +0100 @@ -0,0 +1,79 @@ +{% extends "layout.html" %} +{% if not session["user_logged"] or not session["user_can_edit"][current_repository] %} + {% set readonly="readonly" %} +{% else %} + {% set readonly=False %} +{% endif %} +{% block title %} {{ current_repository}}: Espaces de discussion {% endblock title %} +{% block head %} + {{ super() }} +{% endblock head %} +{% block navbar_items %} + {{ super() }} + {% if session.get("user_logged", None) %} +
  • >
  • +
  • Atelier
  • +
  • >
  • +
  • Social
  • + {% endif %} +{% endblock navbar_items %} +{% block repo_list %} + {{ super() }} +{% endblock repo_list %} +{% block page_content %} +

    CatEdit - {{current_repository}}

    + {% if readonly %} + + {% endif %} +

    Discussions générales

    +

    Créer une nouvelle discussion : + + + +

    + + + + + + + + + + + + {% if not discussions_list %} + + + + {% else %} + {% for discussion in discussions_list %} + + + + + + + + + {% endfor %} + + + + {% endif %} + +
    AuteurDateTitreCommentairesDernier commentaire
    + Il n'y a de discussion à afficher pour cette ensemble de catégories. Créer une discussion +
    {{discussion["author"]}}{{discussion["opening_date"]}}{{discussion["title"]}}{{discussion["comment_count"]}}{{discussion["last_updated"]}} + + +
    + {% import "macros.html" as macros %} + {{ macros.render_pagination(discussions_pagination, discussions_per_page, page_arg_name="discussions_page", per_page_arg_name="discussions_per_page") }} +
    +{% endblock page_content%} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/templates/social/index.html --- a/src/catedit/templates/social/index.html Fri Feb 20 18:42:52 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,118 +0,0 @@ -{% extends "layout.html" %} -{% if not session["user_logged"] or not session["user_can_edit"][current_repository] %} - {% set readonly="readonly" %} -{% else %} - {% set readonly=False %} -{% endif %} -{% block title %} {{ current_repository}}: Espaces de discussion {% endblock title %} -{% block head %} - {{ super() }} -{% endblock head %} -{% block navbar_items %} - {{ super() }} - {% if session.get("user_logged", None) %} -
  • >
  • -
  • Atelier
  • -
  • >
  • -
  • Social
  • - {% endif %} -{% endblock navbar_items %} -{% block repo_list %} - {{ super() }} -{% endblock repo_list %} -{% block page_content %} -

    CatEdit - {{current_repository}}

    - {% if readonly %} - - {% endif %} -

    Historique des modifications

    - - - - - - - - - - - {% if not changeset_list %} - - - - {% else %} - {% for changeset in changeset_list %} - - - - - - - - {% endfor %} - {% endif %} - - - - -
    AuteurDateMessage de soumissionCommentaires
    - Il n'y a aucune modification à afficher. -
    {{changeset["author"]}}{{changeset["date"]}}{{changeset["title"]}}{{changeset["comment_count"]}} - - -
    - {% import "macros.html" as macros %} - {{ macros.render_pagination(commits_pagination, commits_per_page, page_arg_name="commits_page", per_page_arg_name="commits_per_page") }} -
    -

    Discussions générales

    -

    Créer une nouvelle discussion : - - - -

    - - - - - - - - - - - - {% if not discussion_list %} - - - - {% else %} - {% for discussion in discussion_list %} - - - - - - - - - {% endfor %} - - - - {% endif %} - -
    AuteurDateTitreCommentairesDernier commentaire
    - Il n'y a de discussion à afficher pour cette ensemble de catégories. Créer une discussion -
    {{discussion["author"]}}{{discussion["opening_date"]}}{{discussion["title"]}}{{discussion["comment_count"]}}{{discussion["last_updated"]}} - - -
    - {{ macros.render_pagination(discussions_pagination, discussions_per_page, page_arg_name="discussions_page", per_page_arg_name="discussions_per_page") }} -
    -{% endblock page_content%} diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/utils.py --- a/src/catedit/utils.py Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/utils.py Mon Mar 02 14:39:22 2015 +0100 @@ -1,26 +1,134 @@ -from flask.ext.restful import abort -from models import Category, CategoryManager -from rdflib.compare import * - -''' - This function will compare two categories from their Graph object - serializations. - Exact return values are to be defined, some ideas - * check if the 2 categories are the same? (same ID, isomorphic) - * check if the 2 categories are the same but different version? - (same ID, not isomorphic) - * check if unique, non empty parameters changed and list them - (label and description) - * returns 2 lists of properties: only_in_first, only_in_second - * more? -''' +""" + utils.py: + Module that lists utility functions used through the app +""" +from catedit import app +from rdflib import RDF, RDFS +from rdflib.compare import to_isomorphic, graph_diff -def compare_two_cat(cat_serial_1, cat_serial_2): - cat1 = Category(cat_serial_1) - cat2 = Category(cat_serial_2) - iso_cat1 = cat1.cat_graph.to_isomorphic() - iso_cat2 = cat2.cat_graph.to_isomorphic() - if iso_cat1 != iso_cat2: - in_both_cats, only_in_cat1, only_in_cat2 = graph_diff(cat1, cat2) - return +def compare_categories(first_category, second_category, with_details=True): + """ + Compares 2 categories and generate a dict with 3 lists of + differences: + * "same_category" is a True/False attribute which is True if both + categories have the same id + * "same_content" is a True/False attribute which is True if both + categories are the exact same + * "only_in_first" are the properties we can only find + in first_category + * "only_in_second" are the properties we can only find + in second_category + * "in_both" are the properties that can be found in both + categories. To be considered "in both" a property has to have the same + predicate and object in both categories that must have the same id. + + Each "property" is a triple with predicate id (property key defined in + config PROPERTY_LIST) and object (either id if we're referencing + another category or the link/text in other cases) + + If both categories are the exact same, returns + """ + compare_result = {} + first_iso_cat = to_isomorphic(first_category.cat_graph) + second_iso_cat = to_isomorphic(second_category.cat_graph) + + compare_result["same_id"] = ( + first_category.cat_id == second_category.cat_id + ) + compare_result["same_content"] = ( + first_iso_cat == second_iso_cat + ) + + if not(compare_result["same_content"]) and with_details: + rdf_in_both = [] + rdf_only_in_first = [] + rdf_only_in_second = [] + rdf_in_both, rdf_only_in_first, rdf_only_in_second = graph_diff( + first_category.cat_graph, + second_category.cat_graph + ) + in_first = [] + in_both = [] + in_second = [] + for (final_list, diff_list) in [ + (in_both, rdf_in_both), + (in_first, rdf_only_in_first), + (in_second, rdf_only_in_second) + ]: + for triple in diff_list.triples((None, None, None)): + if triple[1] == RDFS.label: + final_list.append(("label", triple[2].toPython())) + elif triple[1] == RDF.Description: + final_list.append(("description", triple[2].toPython())) + else: + for predicate in app.config["PROPERTY_LIST"].keys(): + if triple[1] == \ + app.config["PROPERTY_LIST"][predicate]["rdflib_class"]: + if (app.config["PROPERTY_LIST"] + [predicate] + ["object_type"] == "uriref-link" + or app.config["PROPERTY_LIST"] + [predicate] + ["object_type"] == "literal"): + final_list.append( + ( + predicate, + triple[2].toPython() + ) + ) + else: + final_list.append( + ( + predicate, + triple[2].toPython().split("#", 1)[1] + ) + ) + compare_result["only_in_first"] = in_first + compare_result["only_in_second"] = in_second + compare_result["in_both"] = in_both + print(compare_result) + return compare_result + + + +def make_differences_list(first_category_list, second_category_list): + """ + Compares 2 category lists and generates a dict that lists addition, + modification and deletions from first_category_list to + second_category_list + + * "additions": list of categories that were added + * "modifications" : list of couples before/after of categories that + were modified + * "deletions": list of categories that were deleted + """ + created_categories = [] + modified_categories = [] + deleted_categories = [] + + for first_list_category in first_category_list: + if first_list_category.cat_id not in [second_list_category.cat_id + for second_list_category in second_category_list]: + deleted_categories.append(first_list_category) + else: + for second_list_category in second_category_list: + if first_list_category.cat_id == second_list_category.cat_id: + if not(compare_categories( + first_list_category, + second_list_category, + with_details=False + )["same_content"]): + modified_categories.append( + (first_list_category, second_list_category) + ) + + for second_list_category in second_category_list: + if second_list_category.cat_id not in [first_list_category.cat_id + for first_list_category in first_category_list]: + created_categories.append(second_list_category) + return { + "additions": created_categories, + "modifications": modified_categories, + "deletions": deleted_categories + } diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/views/categories.py --- a/src/catedit/views/categories.py Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/views/categories.py Mon Mar 02 14:39:22 2015 +0100 @@ -3,12 +3,12 @@ The views functions that handle category management and editing """ -from catedit import app, cache +from catedit import app from catedit.models import Category from catedit.views.utils import check_user_status_and_repo_access, \ - get_current_category_list -from flask import render_template, request, redirect, url_for, session, \ - abort, Blueprint + get_current_category_list, get_commits, \ + get_issues +from flask import render_template, request, redirect, url_for, abort, Blueprint from flask_wtf import Form from catedit.resources import CategoryAPI, CategoryChangesAPI from wtforms import StringField, TextAreaField, FormField, \ @@ -77,9 +77,23 @@ cat_list = get_current_category_list(repository=repository) # logger.debug(c.properties) cat_list = sorted(cat_list, key=lambda cat: cat['cat_id']) - return render_template('categories/workshop.html', - cat_list=cat_list, - current_repository=repository) + changeset_list = get_commits( + repository, + 5, + 1 + )[0] + discussion_list = get_issues( + repository, + 5, + 1 + )[0] + return render_template( + 'categories/workshop.html', + cat_list=cat_list, + changeset_list=changeset_list, + discussion_list=discussion_list, + current_repository=repository + ) class CommitForm(Form): diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/views/home.py --- a/src/catedit/views/home.py Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/views/home.py Mon Mar 02 14:39:22 2015 +0100 @@ -4,14 +4,9 @@ """ from catedit import app, github, log_api_rate -from requests import get -from requests.auth import HTTPBasicAuth -from flask import render_template, request, redirect, url_for, \ +from flask import render_template, redirect, url_for, \ session, Blueprint from flask.ext.github import GitHubError -from flask_wtf import Form -from wtforms import StringField, PasswordField -from wtforms.validators import DataRequired module = Blueprint('home', __name__) logger = app.logger diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/views/social.py --- a/src/catedit/views/social.py Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/views/social.py Mon Mar 02 14:39:22 2015 +0100 @@ -6,8 +6,8 @@ from catedit import app from catedit.views.utils import check_user_status_and_repo_access, \ get_comments, post_comment, get_commits, \ - get_issues, get_category_list_for_commit, \ - Pagination + get_issues, get_category_list_for_commit +from catedit.utils import make_differences_list from flask import render_template, request, redirect, url_for, \ abort, Blueprint, session from flask_wtf import Form @@ -19,28 +19,7 @@ @module.route( - "//social", - methods=["GET"], - defaults={ - "commits_page": 1, - "commits_per_page": 10, - "discussions_page": 1, - "discussions_per_page": 10 - } -) -@module.route( - "//social_" - + "commits_page_perpage", - methods=["GET"], - defaults={ - "discussions_page": 1, - "discussions_per_page": 10 - } -) -@module.route( - "//social_" - + "discussions_page" - + "_perpage", + "//changesets", methods=["GET"], defaults={ "commits_page": 1, @@ -48,61 +27,66 @@ } ) @module.route( - "//social_" - + "commits_page_perpage-" - + "discussions_page" - + "_perpage", + "//changesets" + + "/page_perpage", methods=["GET"] ) -def index(repository, commits_page, commits_per_page, - discussions_page, discussions_per_page): +def changesets_index(repository, commits_page, commits_per_page): """ - View that displays every changeset and thread in general discussion - and links to every thread + View that displays every changeset and links to every thread """ check_user_status_and_repo_access(repository) - changeset_list = get_commits( + changeset_list, commits_pagination = get_commits( repository, commits_per_page, commits_page ) - if (changeset_list == [] and commits_page != 1 - and session.get("pagination_links", None) is None): - abort(404) - commits_pagination = None - if session.get("pagination_links", None) is not None: - commits_pagination = Pagination( - page=session["pagination_links"]["current_page"], - per_page=commits_per_page, - last_page=session["pagination_links"]["last_page"] - ) - session.pop("pagination_links", None) - discussion_list = get_issues( + + session.pop("pagination_links", None) + + return render_template( + "social/changesets_index.html", + current_repository=repository, + changeset_list=changeset_list, + commits_pagination=commits_pagination, + commits_per_page=commits_per_page, + ) + +@module.route( + "//discussions", + methods=["GET"], + defaults={ + "discussions_page": 1, + "discussions_per_page": 10, + } +) +@module.route( + "//discussions" + + "/page_perpage", + methods=["GET"] +) +def discussions_index(repository, discussions_page, discussions_per_page): + """ + View that displays thread in general discussion and links to + every thread + """ + check_user_status_and_repo_access(repository) + + discussions_list, discussions_pagination = get_issues( repository, discussions_per_page, discussions_page ) - if discussion_list == [] and discussions_page != 1: + if discussions_list == [] and discussions_page != 1: abort(404) - discussions_pagination = None - if session.get("pagination_links", None) is not None: - discussions_pagination = Pagination( - page=session["pagination_links"]["current_page"], - per_page=discussions_per_page, - last_page=session["pagination_links"]["last_page"] - ) - session.pop("pagination_links", None) return render_template( - "social/index.html", + "social/discussions_index.html", current_repository=repository, - discussion_list=discussion_list, - changeset_list=changeset_list, - commits_pagination = commits_pagination, - commits_per_page = commits_per_page, - discussions_pagination = discussions_pagination, - discussions_per_page = discussions_per_page + discussions_list=discussions_list, + discussions_pagination=discussions_pagination, + discussions_per_page=discussions_per_page, ) @@ -141,7 +125,7 @@ check_user_status_and_repo_access(repository) comment_form = CommentForm() - comments_list = get_comments( + comments_list, pagination = get_comments( repository=repository, thread_type="commits", thread_id=changeset_id, @@ -150,37 +134,43 @@ ) if comments_list == [] and page != 1: abort(404) - pagination=None - if session.get("pagination_links", None) is not None: - # If there are multiple pages we create a pagination class that - # will be sent to the template - pagination = Pagination( - page=session["pagination_links"]["current_page"], - per_page=per_page, - last_page=session["pagination_links"]["last_page"] - ) - session.pop("pagination_links", None) - rdf_old_cat_list = get_category_list_for_commit( - repository, - changeset_id - ) - rdf_new_cat_list = get_category_list_for_commit( + old_category_list = get_category_list_for_commit( repository, changeset_id, get_parent=True ) - old_cat_list = [] - for category in rdf_old_cat_list: - old_cat_list.append({ + new_category_list = get_category_list_for_commit( + repository, + changeset_id + ) + + category_differences_dict = { + list_type: [ + (first_category.label, second_category.label) + for (first_category, second_category) in diff_list + ] if list_type == "modifications" else [ + category.label + for category in diff_list + ] + for list_type, diff_list in make_differences_list( + old_category_list, + new_category_list, + ).items() + } + + old_category_list_template = [] + for category in old_category_list: + old_category_list_template.append({ "cat_label": category.label, "cat_description": category.description, "cat_id": category.cat_id, "cat_properties": category.properties, "state": "original" }) - new_cat_list = [] - for category in rdf_new_cat_list: - new_cat_list.append({ + + new_category_list_template = [] + for category in new_category_list: + new_category_list_template.append({ "cat_label": category.label, "cat_description": category.description, "cat_id": category.cat_id, @@ -190,8 +180,9 @@ if request.method == "GET": return render_template( "social/changeset.html", - old_cat_list=old_cat_list, - new_cat_list=new_cat_list, + old_cat_list=old_category_list_template, + new_cat_list=new_category_list_template, + differences=category_differences_dict, comments=comments_list, changeset_id=changeset_id, comment_form=comment_form, @@ -216,7 +207,8 @@ # Form didn't validate return render_template( "social/changeset.html", - cat_list=cat_list, + old_cat_list=old_category_list_template, + new_cat_list=new_category_list_template, comments=comments_list, changeset_id=changeset_id, comment_form=comment_form, @@ -262,7 +254,7 @@ comments_list = [] else: comment_form = CommentForm() - comments_list = get_comments( + comments_list, pagination = get_comments( repository=repository, thread_type="issues", thread_id=discussion_id, @@ -271,15 +263,6 @@ ) if comments_list == [] and page != 1: abort(404) - if session.get("pagination_links", None) is not None: - # If there are multiple pages we create a pagination class that - # will be sent to the template - comments_pagination = Pagination( - page=session["pagination_links"]["current_page"], - per_page=per_page, - last_page=session["pagination_links"]["last_page"] - ) - session.pop("pagination_links", None) if request.method == "GET": return render_template( @@ -292,28 +275,28 @@ comments_per_page=per_page ) elif request.method == "POST" and comment_form.validate_on_submit(): - if discussion_id == "new": - return_id = post_comment( - repository=repository, - thread_type="issues", - thread_id=discussion_id, - comment_body=comment_form.comment_field.data, - thread_title=comment_form.discussion_title.data - ) - else: - return_id = post_comment( - repository=repository, - thread_type="issues", - thread_id=discussion_id, - comment_body=comment_form.comment_field.data - ) - return redirect(url_for( - "social.discussion", + if discussion_id == "new": + return_id = post_comment( + repository=repository, + thread_type="issues", + thread_id=discussion_id, + comment_body=comment_form.comment_field.data, + thread_title=comment_form.discussion_title.data + ) + else: + return_id = post_comment( repository=repository, - discussion_id=return_id, - page=session["pagination_links"]["last_page"], - per_page=per_page - )) + thread_type="issues", + thread_id=discussion_id, + comment_body=comment_form.comment_field.data + ) + return redirect(url_for( + "social.discussion", + repository=repository, + discussion_id=return_id, + page=session["pagination_links"]["last_page"], + per_page=per_page + )) else: # Form didn't validate return render_template( diff -r 5bd3fb023396 -r ddba4624d661 src/catedit/views/utils.py --- a/src/catedit/views/utils.py Fri Feb 20 18:42:52 2015 +0100 +++ b/src/catedit/views/utils.py Mon Mar 02 14:39:22 2015 +0100 @@ -8,7 +8,7 @@ from catedit import app, github, cache, log_api_rate, save_links from catedit.models import Category from catedit.resources import CategoryAPI, CategoryChangesAPI -from flask import redirect, url_for, session, abort +from flask import redirect, url_for, session from flask.ext.github import GitHubError from datetime import datetime from rdflib import Graph @@ -80,105 +80,120 @@ "date": date of the comment format dd/mm/yy hh:mm } """ - github_comments_data = [] - - try: - github_comments_data = github.get( - "repos/" - + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" - + repository + "/" - + thread_type + "/" - + thread_id - + "/comments?per_page=" + str(per_page) - + "&page=" + str(page), - hooks=dict(response=save_links) - ) - - except GitHubError as ghe: - logger.error( - "Error trying to get comments with following data:" - + " - repository : " + repository - + " - thread_type : " + thread_type - + " - thread_id : " + thread_id - + " - page : " + page - + " - per_page : " + per_page - ) - logger.error(ghe.response.text) - - comment_list = [] - for comment in github_comments_data: - comment_dict = { - "author": comment["user"]["login"], - "body": comment["body"], - "date": convert_github_date( - comment["created_at"] - ) - } - comment_list.append(comment_dict) + cache_key = "get_comments_" \ + + repository + "_" \ + + thread_type + "_" \ + + thread_id + "_" \ + + str(page) + "_" + str(per_page) + if cache.get(cache_key) is None: + github_comments_data = [] - discussion_data = {} - try: - discussion_data = github.get( - "repos/" - + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" - + repository + "/" - + thread_type + "/" - + thread_id, - hooks=dict(response=log_api_rate) - ) - except GitHubError as ghe: - logger.error( - "Error trying to get the or issue of id " + thread_id - ) - logger.error( - "endpoint: " + "repos/" - + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" - + repository + "/" - + thread_type + "/" - + thread_id - ) - logger.error(ghe.response.text) - - thread_author = "" - thread_opening_date = "" - thread_title = "" - thread_opening_post = "" - - route_target = "" - - if thread_type == "commits": - thread_author = discussion_data.get("author", {}).get("login", "") - thread_opening_date = convert_github_date( - discussion_data.get( - "commit", - {} - ).get( - "author", - {} - ).get("date", "") - ) - thread_title = discussion_data.get("commit", {}).get("message", "") - elif thread_type == "issues": - route_target = "social.discussion" - thread_author = discussion_data.get("user", {}).get("login", "") - thread_opening_date = convert_github_date( - discussion_data.get("created_at", "0001-01-01T00:00:00Z") - ) - thread_title = discussion_data.get("title", "") - thread_opening_post = discussion_data.get("body", "") + try: + github_comments_data = github.get( + "repos/" + + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" + + repository + "/" + + thread_type + "/" + + thread_id + + "/comments?per_page=" + str(per_page) + + "&page=" + str(page), + hooks=dict(response=save_links) + ) + except GitHubError as ghe: + logger.error( + "Error trying to get comments with following data:" + + " - repository : " + repository + + " - thread_type : " + thread_type + + " - thread_id : " + thread_id + + " - page : " + page + + " - per_page : " + per_page + ) + logger.error(ghe.response.text) + pagination = None + if session.get("pagination_links", None) is not None: + # If there are multiple pages we create a pagination class that + # will be sent to the template + pagination = Pagination( + page=session["pagination_links"]["current_page"], + per_page=per_page, + last_page=session["pagination_links"]["last_page"] + ) + session.pop("pagination_links", None) - thread_dict = { - "author": thread_author, - "title": thread_title, - "opening_date": thread_opening_date, - "comment_list": comment_list, - "opening_post": thread_opening_post, - "per_page": per_page - } + comment_list = [] + for comment in github_comments_data: + comment_dict = { + "author": comment["user"]["login"], + "body": comment["body"], + "date": convert_github_date( + comment["created_at"] + ) + } + comment_list.append(comment_dict) + + discussion_data = {} + try: + discussion_data = github.get( + "repos/" + + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" + + repository + "/" + + thread_type + "/" + + thread_id, + hooks=dict(response=log_api_rate) + ) + except GitHubError as ghe: + logger.error( + "Error trying to get the commit or issue of id " + thread_id + ) + logger.error( + "endpoint: " + "repos/" + + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" + + repository + "/" + + thread_type + "/" + + thread_id + ) + logger.error(ghe.response.text) - return thread_dict + thread_author = "" + thread_opening_date = "" + thread_title = "" + thread_opening_post = "" + if thread_type == "commits": + thread_author = discussion_data.get("author", {}).get("login", "") + thread_opening_date = convert_github_date( + discussion_data.get( + "commit", + {} + ).get( + "author", + {} + ).get("date", "") + ) + thread_title = discussion_data.get("commit", {}).get("message", "") + elif thread_type == "issues": + thread_author = discussion_data.get("user", {}).get("login", "") + thread_opening_date = convert_github_date( + discussion_data.get("created_at", "0001-01-01T00:00:00Z") + ) + thread_title = discussion_data.get("title", "") + thread_opening_post = discussion_data.get("body", "") + + + thread_dict = { + "author": thread_author, + "title": thread_title, + "opening_date": thread_opening_date, + "comment_list": comment_list, + "opening_post": thread_opening_post, + "per_page": per_page + } + + cache.set(cache_key, (thread_dict, pagination), timeout=3600) + return (thread_dict, pagination) + else: + return cache.get(cache_key) def post_comment(repository, thread_type, thread_id, comment_body, thread_title=""): @@ -256,33 +271,48 @@ comment_count : commit comments count } """ - commits_data = [] - try: - commits_data = github.get( - "repos/" - + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" - + repository - + "/commits?per_page=" + str(per_page) + "&page=" + str(page), - hooks=dict(response=save_links) - ) - except GitHubError as ghe: - logger.error("Error getting commits for repo " + repository) - logger.error(ghe.response.text) - changeset_list = [ - { - "id": commit["sha"], - "title": commit["commit"]["message"], - "date": convert_github_date( - commit["commit"]["committer"]["date"] - ), - "author": commit["commit"]["committer"]["name"], - "comment_count": commit["commit"]["comment_count"], - } - for commit in commits_data - ] + cache_key = "get_commits_" \ + + repository + "_" \ + + str(page) + "_" + str(per_page) + if cache.get(cache_key) is None: + commits_data = [] + try: + commits_data = github.get( + "repos/" + + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" + + repository + + "/commits?per_page=" + str(per_page) + "&page=" + str(page), + hooks=dict(response=save_links) + ) + except GitHubError as ghe: + logger.error("Error getting commits for repo " + repository) + logger.error(ghe.response.text) - return changeset_list + commits_pagination = None + if session.get("pagination_links", None) is not None: + commits_pagination = Pagination( + page=session["pagination_links"]["current_page"], + per_page=per_page, + last_page=session["pagination_links"]["last_page"] + ) + session.pop("pagination_links", None) + changeset_list = [ + { + "id": commit["sha"], + "title": commit["commit"]["message"], + "date": convert_github_date( + commit["commit"]["committer"]["date"] + ), + "author": commit["commit"]["committer"]["name"], + "comment_count": commit["commit"]["comment_count"], + } + for commit in commits_data + ] + cache.set(cache_key, (changeset_list, commits_pagination), timeout=3600) + return (changeset_list, commits_pagination) + else: + return cache.get(cache_key) def get_issues(repository, per_page=30, page=1): """ @@ -297,31 +327,47 @@ comment_count: comments count } """ - issues_data = [] - try: - issues_data = github.get( - "repos/" - + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" - + repository - + "/issues?per_page=" + str(per_page) + "&page=" + str(page), - hooks=dict(response=save_links) - ) - except GitHubError as ghe: - logger.error("Error getting issues for repo " + repository) - logger.error(ghe.response.text) + cache_key = "get_issues_" \ + + repository + "_" \ + + str(page) + "_" + str(per_page) + if cache.get(cache_key) is None: + issues_data = [] + try: + issues_data = github.get( + "repos/" + + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" + + repository + + "/issues?per_page=" + str(per_page) + "&page=" + str(page), + hooks=dict(response=save_links) + ) + except GitHubError as ghe: + logger.error("Error getting issues for repo " + repository) + logger.error(ghe.response.text) - discussion_list = [ - { - "id": str(issue["number"]), - "title": issue["title"], - "author": issue["user"]["login"], - "opening_date": convert_github_date(issue["created_at"]), - "last_updated": convert_github_date(issue["updated_at"]), - "comment_count": issue["comments"], - } - for issue in issues_data - ] - return discussion_list + discussions_pagination = None + if session.get("pagination_links", None) is not None: + discussions_pagination = Pagination( + page=session["pagination_links"]["current_page"], + per_page=per_page, + last_page=session["pagination_links"]["last_page"] + ) + session.pop("pagination_links", None) + + discussion_list = [ + { + "id": str(issue["number"]), + "title": issue["title"], + "author": issue["user"]["login"], + "opening_date": convert_github_date(issue["created_at"]), + "last_updated": convert_github_date(issue["updated_at"]), + "comment_count": issue["comments"], + } + for issue in issues_data + ] + cache.set(cache_key, (discussion_list, discussions_pagination), timeout=3600) + return (discussion_list, discussions_pagination) + else: + return cache.get(cache_key) def get_category_list_for_commit(repository, changeset_id, get_parent=False): @@ -329,79 +375,91 @@ Get the category list as it was following the changeset of id changeset_id """ - - # First step - commit_data = {} - try: - commit_data = github.get( - "repos/" - + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" - + repository + "/commits/" - + changeset_id - ) - except GitHubError as ghe: - logger.error("Error trying to get the commit of id " + changeset_id) - logger.error(ghe.response.text) - - if get_parent: - parents = commit_data.get("parents", []) - if parents != []: - parent_sha = parents[0].get("sha", "") + cache_key = "get_category_list_for_commit_" \ + + repository + "_" \ + + changeset_id + "_parent_" + str(get_parent) + if cache.get(cache_key) is None: + # First step + commit_data = {} try: commit_data = github.get( "repos/" + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" + repository + "/commits/" - + parent_sha + + changeset_id ) except GitHubError as ghe: - logger.error("Error trying to get the commit of id " + parent_sha) + logger.error( + "Error trying to get the commit of id " + changeset_id + ) logger.error(ghe.response.text) - tree_sha = commit_data.get("commit", {}).get("tree", {}).get("sha", "") - - # Second step - tree_data = {} - try: - tree_data = github.get( - "repos/" - + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" - + repository + "/git/trees/" - + commit_data["commit"]["tree"]["sha"] - + "?recursive=1" - ) - except GitHubError as ghe: - logger.error("Error trying to get the tree of sha " + tree_sha) - logger.error(ghe.response.text) - - # Third step and fourth step - cat_list = [] - for blob in tree_data.get("tree", []): - if app.config["PERSISTENCE_CONFIG"]["CATEGORIES_PATH"] in blob["path"]: - blob_data = {} + parent_sha = "" + if get_parent: + parents = commit_data.get("parents", []) + if parents != []: + parent_sha = parents[0].get("sha", "") try: - blob_data = github.get( + commit_data = github.get( "repos/" + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] - + "/" + repository + "/git/blobs/" - + blob["sha"] + + "/" + repository + "/commits/" + + parent_sha ) except GitHubError as ghe: logger.error( - "Error trying to get the blob of sha " + blob["sha"] + "Error trying to get the commit of id " + parent_sha ) logger.error(ghe.response.text) - cat_graph = Graph() - cat_graph.parse( - source=StringIO( - str(b64decode(blob_data["content"]), "utf-8") - ), - format="turtle" + tree_sha = commit_data.get("commit", {}).get("tree", {}).get("sha", "") + + # Second step + tree_data = {} + try: + tree_data = github.get( + "repos/" + + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + "/" + + repository + "/git/trees/" + + commit_data["commit"]["tree"]["sha"] + + "?recursive=1" ) - category = Category(graph=cat_graph) - cat_list.append(category) - return cat_list + except GitHubError as ghe: + logger.error("Error trying to get the tree of sha " + tree_sha) + logger.error(ghe.response.text) + + # Third step and fourth step + cat_list = [] + for blob in tree_data.get("tree", []): + if app.config["PERSISTENCE_CONFIG"]["CATEGORIES_PATH"] \ + in blob["path"]: + blob_data = {} + try: + blob_data = github.get( + "repos/" + + app.config["PERSISTENCE_CONFIG"]["REPOSITORY_OWNER"] + + "/" + repository + "/git/blobs/" + + blob["sha"] + ) + except GitHubError as ghe: + logger.error( + "Error trying to get the blob of sha " + blob["sha"] + ) + logger.error(ghe.response.text) + + cat_graph = Graph() + cat_graph.parse( + source=StringIO( + str(b64decode(blob_data["content"]), "utf-8") + ), + format="turtle" + ) + category = Category(graph=cat_graph) + cat_list.append(category) + cache.set(cache_key, cat_list, timeout=3600) + return cat_list + else: + return cache.get(cache_key) def convert_github_date(date): @@ -430,6 +488,9 @@ "state": category state (one of {"untouched", "created", "edited", "deleted"}) } + + Caching it is unecessary because the API call to get the list is + already cached """ cat_api_instance = CategoryAPI() cat_changes_api_instance = CategoryChangesAPI() @@ -486,10 +547,6 @@ # now we must find the not yet submitted categories that were created cat_state = "" - logger.debug("Edited cat list: " - + str([cat.label for cat in edited_cat_list]) - + " - Original cat list: " - + str([cat.label for cat in original_cat_list])) for category in edited_cat_list: if category.cat_id not in [cat.cat_id for cat in original_cat_list]: