\chapter{Evaluation}
\label{eval}

We tried to asses the quality of ICFS by the following metrics:

\begin{itemize}
	\item Security : Does ICFS effectively mitigate unauthorized access by untrusted processes?
	\item Usability : Does the interactive model reduce configuration burden while maintaining user control?
	\item Performance : What is the overhead introduced by ICFS compared to native filesystem operations?
	\item Compatibility : How well does ICFS integrate with existing workflows and software (e.g., CLI tools, legacy applications)?
\end{itemize}

\section{Test environment}

Performance and usability evaluations were conducted on an HP Pavilion Laptop 15-cc563st equipped with an Intel® Core™ i7-7500U processor, a Western Digital WDS250G2B0B WD Blue 3D NAND M.2 SATA SSD (250GB), and 12 GB of DDR4 RAM. The system ran Fedora Linux 42 (Workstation Edition) with kernel version 6.14.5-300.fc42.x86\_64 and GNOME 48 under the Wayland session. For additional compatibility testing, a KVM virtual machine hosted on the same hardware emulated a Debian GNU/Linux 12 (bookworm) environment with kernel 6.1.0-27-amd64, GNOME 43.9, and the X11 windowing system. The virtual machine had 2 CPU cores and 2 GB of RAM.

To simulate real-world usage, ICFS was mounted to a directory pre-populated with files, and interactions were tested across six widely used applications:

\begin{itemize}
	\item {\TeX}studio.
	\item Programming IDE/text editor (Neovim).
	\item A web browser (Mozilla Firefox).
	\item File manager (GNOME Files, also known as Nautilus).
	\item File synchronisation software (Syncthing).
	\item Standard command line core utilities (e.g. \verb|ls|, \verb|cd|, \verb|grep|, ...).
\end{itemize}

These tools were selected for their prevalence and diverse filesystem interaction patterns. Except for Syncthing and Firefox (distributed via Flatpak containers), all applications were installed as native packages. 

\iffalse

For performance testing, special scripts were written, that test the speed of `open` operations. We did not test other operations, since 

\fi

\todo[inline, caption={Complete Evaluation}, author={\textbf{Draft note}}]{The rest of this chapter is \emph{very} incomplete, and only contains a brief and informal talk about the issues I am facing right now. This is not the actual thesis-worthy text. All issues discussed were relevant as of 13.05.2025} 

\section{Security}

\todo[inline, author={\textbf{Draft note}}]{What to even say here?}

\section{Usability}

Testing revealed that most applications interacted smoothly with ICFS. TeXstudio and Neovim required a single permission prompt when opening existing projects, with no further interruptions for new file creation.

GNOME Files (Nautilus) exhibited unexpected behaviour. While functional without access permissions, its thumbnail generation process triggered repeated prompts. The file manager employs an external utility, gdk-pixbuf-thumbnailer, to create previews of files such as images. 

\begin{figure}[H]
	\centering
	\includegraphics[width=0.6\linewidth]{./images/picture-preview-nautilus.png}
	\caption{Example of a picture mini-preview: image on the left is the scaled-down version of the actual image stored in the file.}
\end{figure}

Each preview required individual authorisation because the thumbnailer operates per-file, invalidating temporary folder permissions after each new image access. A pragmatic solution involves granting permanent access to the thumbnailer for specific directories -- a trade-off between convenience and security.

Firefox interactions were limited to file downloads and uploads, which utilise system-level file selection dialogs managed by the system file manager.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.6\linewidth]{./images/file-selection-dialogue.png}
	\caption{File selection dialogue in Firefox.}
\end{figure}

Hence, it inherited all issues Nautilus had. However, denying Nautilus permissions did not disrupt the download process itself, and ensured users could selectively authorise only the target save location -- a critical safeguard for internet-facing applications.

Containerisation significantly impacted Syncthing ’s usability. Flatpak-packaged applications often modify filesystem paths visible within their sandbox. In Syncthing’s case, the executable’s relocated path invalidated \verb|/proc/pid/exe| -- a symbolic link in Linux’s procfs filesystem that typically points to a process’s binary. ICFS relies on this link to verify process identities for persistent permissions.

\section{Performance}

Performance of ICFS is terrible. \sout{Unfortunately, I was unable to make perf work with it for some reason, so I don't really know what is slowing operations down. So those are my speculations for what \emph{may} be the bottleneck. A lot of it is caused by it's design. For example, ICFS needs to look through procfs to get process creation time, and there is no way of going around this it seems.}

I managed to get the \verb|perf| to work: it showed that almost all time was consumed by libfuse, not my program. My code used something like 0.0001\% of runtime on a pretty heavy test. I am not quite sure what to do, since libfuse is already optimised to smithereens. I guess I will just write how it is and not touch the performance ever again.

\section{Limitations}
\label{eval:limit}
