Trial results, cost and risks
Confidentiality trials (November 2026)
Each tool was installed and a logged-in student tried every route found to read mentor feedback, including the API directly.
| Tool | Result |
|---|---|
| NocoBase 2.2 | Failed. Direct access was refused, but a student could read all feedback by requesting it as part of a related record. |
| Frappe Framework 16 | Passed. Direct access, search, reports, joined queries, comments, change history and file attachments were all refused. |
Two limits found in Frappe, and how the design avoids them
- Hiding one field inside a readable record is not reliable (old values show in the change history). Confidential data therefore always has its own record type that students cannot open at all.
- A count leaks. A student can see how many hidden records are linked to their team, but nothing about them. Accepted.
Feature checks
| Check | Result |
|---|---|
| Mentors cannot see each other's scores; customers isolated from each other | Pass, once "strict user permissions" is switched on |
| Workflow states restricted by role | Pass |
| Approve a report from an e-mail link | The link works only after the recipient logs in |
| PDFs of records (merged feedback, reports) | Pass, after changing the PDF generator and giving the server a URL it can reach; access rules apply to PDFs too |
| E-mail (sign-off requests) | Pass, with the scheduler running |
| Milestone Gantt across all teams | Renders (it had not been seen on screen before this prototype); see the tour |
| Kanban, number cards, team-by-week grid | Pass (grid with fixed week columns) |
| Save and restore a checkpoint | About 5 seconds to save, 20 to restore |
| Open the site as another user (administrator) | Pass; the user is notified |
Learned while building the prototype
- Five short server-side scripts are needed (still no custom application): dates from the course clock, checking who may sign off, returning a report when changes are requested, submitting from the portal, and recording who approved feedback.
- The portal (what students and customers use) has no workflow buttons, so each form has a "submit now" tick box.
- Students stay isolated only while each has exactly one team restriction; the test checks this at every checkpoint.
- An unrestricted customer role would have let a customer whose proposal was declined read every sealed specification; customers therefore get a second role only when their project is accepted, always tied to that project.
Cost and effort
| Item | Cost |
|---|---|
| Software licences | None. Frappe Framework is open source (MIT) with no paid edition |
| Hosting | None additional: the coordinator's existing server. Frappe used about 550 MB of memory when idle in the trial |
| AI usage | An estimated $50-150 per semester for 10 teams over 15 weeks on a mid-tier model. The prototype itself spent nothing: its AI outputs are recorded |
Effort, as rough estimates: starting a new semester about half a day; operating it one to two hours a week for the coordinator, more around presentation days.
Risks
| Risk | Mitigation |
|---|---|
| Student data sits on a personally run server | Daily backups, a documented restore, a handover note; move to university hosting if it becomes available |
| The process depends on one coordinator | Everything is in two places that can be handed over: the site's database and the playbook repository |
| Frappe takes more setup than simpler tools; its staff screens are plain | Accept for the pilot; students and customers see only simple portal pages |
| A later change opens a confidentiality hole | The scripted test runs every semester and after every upgrade |
| AI summaries are wrong | Source links on every statement; a mentor approves anything that reaches students |
| Customers give requirements away or forget what was agreed | A one-page playbook; the sealed specification is versioned and the customer can see it |
| Mentors skip the weekly check-in | It takes two minutes; the dashboard shows when one is missing |