Update 9:15am 26/10/20 Drupal CMS publishing issues - database restored 2:40pm 26 October 2020
27 October 2020
Update: 9:15 27 October 2020
The UCL website is fully cached to improve performance. Rebuilding the cache has taken a number of hours yesterday and last night and is now almost complete. The Drupal CMS is now stable, following yesterday's intermittent performance issues, and editors can edit as normal although we would ask you to look out for any issues.
If you do encounter issues, please try first republishing the node affected and then contact firstname.lastname@example.org if that doesn't resolve the problem.
Update: 2:45pm 26 October 2020
We have successfully restored the data however this has caused some issues for the CMS. This is because all content is being added to the web cache again, which takes time and is still continuing, so you may continue to see intermittent issues for the rest of today.
All content has now been reset to its state as of 10pm on 21 October: any content changes made since then will need to be recreated.
If you experience any issues with new or existing content please try first republishing and then contact email@example.com if that doesn't resolve the problem
Many thanks for your patience while the teams involved worked to restore the Drupal databases.
Update 9:45am 26/10/20: Drupal database issues: please do not edit between 10.00 and 10.30am today
We are now ready to restore the Drupal databases. Please do not add or edit content between 10.00 and 10.30 this morning as your changes will be lost.
Once data is restored, all content in the CMS will be reset to its state at 10pm on 21 October so new content and any changes made between then and now will need to be recreated once we give the go-ahead at 10.30.
Update 4:30pm 23 October 2020
Work to restore the missing Drupal data is continuing but we won't be restoring the data until Monday morning at the earliest.
Please continue to keep non-urgent content edits to a minimum as anything you've added since Wednesday 21 October at 10pm will need to be recreated. You may also experience glitches when adding content (particularly News, Events, etc.) so it's best to refrain if at all possible.
We will provide a further update on Monday morning. Thanks for your continued patience.
Update 12 noon 23 October
The recovery of missing Drupal data is proving complex and we don't yet have an estimate for when the data restore will take place. It is likely that the data will not be restored before the weekend but we will provide a further update later this afternoon.
The advice regarding adding new Drupal content remains the same as the previous update.
Please keep non-urgent content edits to a minimum as anything you've added since Wednesday 21 October at 10pm will need to be recreated. You may also experience glitches when adding content (particularly News, Events, etc.) so it's best to refrain if at all possible.
Please do contact firstname.lastname@example.org if you need clarification on the above or have concerns about urgent updates.
Update 5:30pm 22 October 2020
The restore of missing content is taking longer to achieve than expected. We are still working on the solution to the issue and hope that the missing content will be restored tomorrow but we can't give an estimated time at the moment.
The advice regarding adding new Drupal content remains the same as detailed below. Please keep non-urgent content edits to a minimum.
We apologise for the ongoing disruption to the service and we will provide a further update tomorrow morning.
Due to fault with an overnight technical process Drupal content created between 6pm Monday 19 October and Wednesday 21 October at around 10pm is currently missing from all UCL Drupal sites.
We are working to restore all the missing content from a back-up taken yesterday evening at 10pm.
If you have content that needs to be restored urgently e.g. because the link has been shared in a newsletter, then you can recreate this content now at the original URL. It will be overwritten when the data is restored but as long as the URL and content are the same, it will continue to work even after the data restore.
Note that this only applies to content that was created before 10pm last night. Any brand-new content created after 10pm last night will be lost when the missing data is restored and will need to be recreated after that point.
We would therefore advise that only urgent content already created between 19 and 21 October is recreated today and only in cases where a URL has been widely shared e.g via a staff or student email or other communications.
We will post an update as soon as we have more information.
Please accept our apologies for the disruption and do contact email@example.com if you have any questions.