MarketWinFS
Company Profile

WinFS

WinFS was the code name for a canceled data storage and management system project based on relational databases, developed by Microsoft and first demonstrated in 2003. It was intended as an advanced storage subsystem for the Microsoft Windows operating system, designed for persistence and management of structured, semi-structured and unstructured data.

Motivation
Many filesystems found on common operating systems, including the NTFS filesystem which is used in modern versions of Microsoft Windows, store files and other objects only as a stream of bytes, and have little or no information about the data stored in the files. Such file systems also provide only a single way of organizing the files, namely via directories and file names. Because a file system has no knowledge about the data it stores, Using common file formats is a workaround to this problem but not a universal solution; there is no guarantee that all applications will use the format. Data with standardized schema, such as XML documents and relational data fare better, as they have a standardized structure and run-time requirements. Also, a traditional file system can retrieve and search data based only on the filename, because the only knowledge it has about the data is the name of the files that store the data. A better solution is to tag files with attributes that describe them. Attributes are metadata about the files such as the type of file (such as document, picture, music, creator, etc.). This allows files to be searched for by their attributes, in ways not possible using a folder hierarchy, such as finding "pictures which have person X". The attributes can be recognizable by either the file system natively, or via some extension. Desktop search applications take this concept a step further. They extract data, including attributes, from files and index it. To extract the data, they use a filter for each file format. This allows for searching based on both the file's attributes and the data in it. However, this still does not help in managing related data, as disparate items do not have any relationships defined. For example, it is impossible to search for "the phone numbers of all persons who live in Acapulco and each have more than 100 appearances in my photo collection and from whom I have had e-mail within the last month". Such a search could not be done unless it is based on a data model which has both the semantics as well as relationships of data defined. WinFS aims to provide such a data model and the runtime infrastructure that can be used to store the data, as well as the relationships between data items according to the data model, doing so at a satisfactory level of performance. ==Overview==
Overview
WinFS natively recognizes different types of data, such as picture, e-mail, document, audio, video, calendar, contact, rather than just leaving them as raw unanalyzed bytestreams (as most file systems do). Data stored and managed by the system are instances of the data type recognized by the WinFS runtime. The data are structured by means of properties. For example, an instance of a résumé type will surface the data by exposing properties, such as Name, Educational Qualification, Experience. Each property may be a simple type (strings, integers, dates) or complex types (contacts). Different data types expose different properties. Besides that, WinFS also allows different data instances to be related together; such as a document and a contact can be related by an Authored By relationship. Thus WinFS can reduce redundancies. Access to all the data in the system allows complex searches for data across all the data items managed by WinFS. In the example used above ("the phone numbers of all persons who live in Acapulco and each have more than 100 appearances in my photo collection and with whom I have had e-mail within last month"), WinFS can traverse the subject relationship of all the photos to find the contact items. Similarly, it can filter all emails in last month and access the communicated with relation to reach the contacts. The common contacts can then be figured out from the two sets of results and their phone number retrieved by accessing the suitable property of the contact items. In addition to fully schematized data (like XML and relational data), WinFS supports semi-structured data (such as images, which have an unstructured bitstream plus structured metadata) as well as unstructured data (such as files) as well. It stores the unstructured components as files while storing the structured metadata in the structured store. Internally, WinFS uses a relational database to manage data. It does not limit the data to belonging to any particular data model. The WinFS runtime maps the schema to a relational modality, by defining the tables it will store the types in and the primary keys and foreign keys that would be required to represent the relationships. WinFS includes mappings for object and XML schemas by default. Mappings for other schemas must be specified. Object schemas are specified in XML; WinFS generates code to surface the schemas as .NET classes. ADO.NET can be used to directly specify the relational schema, though a mapping to the object schema must be provided to surface it as classes. Relationship traversals are performed as joins on these tables. WinFS also automatically creates indexes on these tables, to enable fast access to the information. Indexing speeds up joins significantly, and traversing relationships to retrieve related data is performed very fast. Indexes are also used during information search; searching and querying use the indexes to quickly complete the operations, much like desktop search systems. == Development ==
Development
The development of WinFS is an extension to a feature that was initially planned in the early 1990s. Dubbed Object File System, it was supposed to be included as part of Cairo. OFS was supposed to have powerful data aggregation features, but the Cairo project was shelved, and with it OFS. However, later during the development of COM, a storage system, called Storage+, based on then-upcoming SQL Server 8.0, was planned, which was slated to offer similar aggregation features. and build 4051 of Windows Vista, then called by its codename "Longhorn", given to developers at the Microsoft Professional Developers Conference in 2003, included WinFS, but it suffered from significant performance issues. The beta was refreshed on December 1, 2005 to be compatible with version 2.0 of the .NET Framework. WinFS Beta 2 was planned for some time later in 2006, and was supposed to include integration with Windows Desktop Search, so that search results include results from both regular files and WinFS stores, as well as allow access of WinFS data using ADO.NET. On June 23, 2006, the WinFS team at Microsoft announced that WinFS would no longer be delivered as a separate product, and some components would be brought under the umbrella of other technologies. Many of the principle features Microsoft intended to provide with WinFS included a pane for metadata property editing, breadcrumb-based property navigation, filtering or stacking items over properties, incremental search, and saved searches; these features were incorporated in Windows Vista. Query composition, a feature of WinFS that allowed users to perform additional searches that reuse the results of a previous query, Examples of uses of the technology are the object-relational mapping components into ADO.NET Entity Framework; support for unstructured data, adminless mode of operation, support for file system objects via the FILESTREAM data type, and hierarchical data in SQL Server 2008, then codenamed Katmai, as well as integration with Win32 APIs and Windows Shell and support for traversal of hierarchies by traversing relationships into later releases of Microsoft SQL Server; ==Data storage==
Data storage
Architecture WinFS uses a relational engine, which derives from SQL Server 2005, to provide the data-relations mechanism. WinFS stores are simply SQL Server database (.MDF) files with the FILESTREAM attribute set. These files are stored in the access-restricted folder named "System Volume Information" (placed in the volume root), in folders under the folder "WinFS" with names of GUIDs of these stores. • WinFS.exe, which hosts the relational datastore • WinFSSearch.exe, which hosts the indexing and querying engine • WinFPM.exe (WinFS File Promotion Manager), which interfaces with the underlying file-system It allows programmatic access to its features via a set of .NET Framework APIs. These enable applications to define custom-made data types, define relationships among data, store and retrieve information, and allow advanced searches. Relations can also be specified by other applications or the user. WinFS provides unified storage, but stops short of defining the format that is to be stored in the data stores. Instead it supports writing data in application-specific formats. But applications must provide a schema that defines how the file format should be interpreted. The Contact Item defined in this way will be used to store information regarding the Contact, by populating the properties field and storing it. Only those fields marked as mandatory needs to be filled up during initial save. which are executed when a certain condition is met. WinFS rules work on data and data relationships. For example, a rule can be created that states that whenever an Item is created which contains field "Name" and if the value of that field is some particular name, a relationship should be created that relates the Item with some other Item. WinFS rules can also access any external application. For example, a rule can be built which launches a Notify application whenever a mail is received from a particular contact. WinFS rules can also be used to add new properties fields to existing data Items. WinFS rules are also exposed as .NET CLR objects. As such any rule can be used for any purpose. A rule can even be extended by inheriting from it to form a new rule that consists of the condition and action of the parent rule plus something more. RAV WinFS supports creating Rich Application Views (RAV) by aggregating different data in a virtual table format. Unlike database view, where each individual element can only be a scalar value, RAVs can have complex Items or even collections of Items. The actual data can be across multiple data types or instances and can even be retrieved by traversing relationships. RAVs are intrinsically paged (dividing the entire set of data into smaller pages containing disconnected subsets of the data) by the WinFS runtime. The page size is defined during creation of the view and the WinFS API exposes methods to iterate over the pages. RAVs also supports modification of the view according to different grouping parameters. Views can also be queried against. Access control Even though all data are shared, everything is not equally accessible. WinFS uses the Windows authentication system to provide two data protection mechanisms. First, there is share-level security that controls access to your WinFS share. Second, there is item level security that supports NT compatible security descriptors. The process accessing the item must have enough privileges to access it. Also in Vista there is the concept of "integrity level" for an application. Higher integrity data cannot be accessed by a lower integrity process. ==Data retrieval==
Data retrieval
The primary mode of data retrieval from a WinFS store is querying the WinFS store according to some criteria, The data in them can be accessed by accessing the properties of individual objects. A set of all matches is returned, which can then be bound to a UI widget for displaying en masse or enumerating individually. The properties items can also be modified and then stored back to the data store to update the data. The ItemContext object is closed (which marks the end of association of the object with the store) when the queries are made or changes merged into the store. Related items can also be accessed through the items. The IncomingRelationships and OutgoingRelationships properties give access to all the set of relationship instances, typed to the name of the relationship. These relationship objects expose the other item via a property. So, for example, if a picture is related to a picture, it can be accessed by traversing the relationship as: ContactsCollection contacts = picture.OutgoingRelationships.Cast(typeof(Contact)).Value; // This retrieves the collection of all outgoing relationships from a picture object // and filters down the contacts reachable from them and retrieves its value. // Or the relationship can be statically specified as ContactsCollection contacts = picture.OutgoingRelationships.OutContactRelationship.Contact; An OPath query string allows to express the parameters that will be queried for to be specified using Item properties, embedded Items as well as Relationships. It can specify a single search condition, such as ''"title = Something'", or a compound condition such as "title = 'Title 1' || title = 'Title 2' && author = 'Someone'". These Boolean and relational operations can be specified using C# like &&, ||, =, != operators as well as their English-like equivalent like EQUAL, NOT EQUAL. SQL like operators such as LIKE, GROUP BY and ORDER BY are also supported, as are wildcard conditions. So, "title LIKE 'any*'"'' is a valid query string. These operators can be used to execute complex searches such as using (ItemContext ic = ItemContext.Open()) { // Searching begins by creating a ItemSearcher object. The searcher is created from a // relationship instance because the contacts being searched for are in relation. The // first parameter defines the scope of the search. An ItemContext as the scope means // the entire store is to be searched. Scope can be limited to a set of Items which may // be in a holding relationship with the contacts. In that case, the set is passed as // the scope of the search. ItemSearcher searcher = OutContactRelationship.GetTargetSearcher(ic, typeof(Contact)); ContactCollection contacts = searcher.FindAll("OutContactRelationship.Contact.Name LIKE 'A*'"); } The above code snippet creates an ItemSearcher object that searches on the OutContactRelationship instance that relates pictures and contacts, in effect searching all pictures related with a contact. It then runs the query ''Name LIKE 'A*'" on all contacts reachable through OutContactRelationship, returning the list of "contacts whose names start with A and whose pictures I have". Similarly, more relationships could be taken into account to further narrow down the results. Further, a natural language query processor, which parses query in natural language and creates a well-formed OPath query string to search via proper relationships, can allow users to make searches such as "find the name of the wine I had with person X last month"'', provided financial management applications are using WinFS to store bills. Different relations specify a different set of data. So when a search is made that encompasses multiple relations, the different sets of data are retrieved individually and a union of the different sets is computed. The resulting set contains only those data items that correspond to all the relations. Notifications WinFS includes better support for handling data that changes frequently. Using WinFS Notifications, applications choose to be notified of changes to selected data Items. WinFS will raise an ItemChangedEvent, using the .NET Event model, when a subscribed-to Item changes, and the event will be published to the applications. Information Agent WinFS includes an Information Agent feature for the management, retrieval, and storage of end-user notification rules and preferences for changes to items in the data store. Using Information Agent, it is possible to automatically define relations to new items based on events such as appointments, with an example being that appointments can be related to photos based on the dates the photos were taken, enabling queries for birthdays or holidays without needing to know the actual dates of such events ("find all photos taken on this birthday"). Other examples include automatically moving new items to specific folders based on a rule as determined by appointment times and dates the photos were taken ("when I import a photo taken during a business event, move it to the Business Events folder") or more complex possibilities. Information Agent can also forward notifications to other devices ("if I receive a high priority email from my boss, send a notification to my phone") and is similar to Rules and Alerts functionality of Microsoft Outlook. ==Data sharing==
Data sharing
WinFS allows easy sharing of data between applications, and among multiple WinFS stores, which may reside on different computers, by copying to and from them. A WinFS item can also be copied to a non-WinFS file system, but unless that data item is put back into the WinFS store, it will not support the advanced services provided by WinFS. The WinFS API also provides some support for sharing with non-WinFS applications. WinFS exposes a shell object to access WinFS stores. This object maps WinFS items to a virtual folder hierarchy, and can be accessed by any application. Virtual folders can automatically share new content referenced by the query with users (a virtual folder for "all vacation photos" can automatically share new items returned by this query with users). WinFS data can also be manually shared using network shares, by sharing the legacy shell object. Non-WinFS file formats can be stored in WinFS stores, using the File Item, provided by WinFS. Importers can be written, to convert specific file formats to WinFS Item types. In addition, WinFS provides services to automatically synchronize items in two or more WinFS stores, subject to some predefined condition, such as "share only photos" or "share photos that have an associated contact X". The stores may be on different computers. Synchronization is done in a peer-to-peer fashion; there is no central authority. A synchronization can be either manual or automatic or scheduled. During synchronization, WinFS finds the new and modified Items, and updates accordingly. If two or more changes conflict, WinFS can either resort to automatic resolution based on predefined rules, or defer the synchronization for manual resolution. WinFS also updates the schemas, if required. ==Application support==
Application support
Shell namespace WinFS Beta 1 includes a shell namespace extension, which surfaces WinFS stores as top level objects in My Computer view. which allowed one to browse WinFS stores by presenting a hierarchical view of WinFS Items. It automatically generated virtual folders based on access permissions, date and other metadata, and presented them in a hierarchical tree view, akin to what traditional folders are presented in. The application generated tabs for different Item types. StoreSpy allowed viewing Items, Relationships, MultiSet, Nested Elements, Extensions and other types in the store along with its full metadata. It also presented a search interface to perform manual searches, and save them as virtual folders. The application also presented a graphical view of WinFS Rules. However, it did not allow editing of Items or their properties, though it was slated for inclusion in a future release. But the WinFS project was cut back before it could materialize. Type Browser WinFS also includes another application, named WinFS Type Browser, which can be used to browse the WinFS types, as well as visualize the hierarchical relationship between WinFS types. A WinFS type, both built-in types as well as custom schemas, can be visualized along with all the properties and methods that it supports. It also shows the types that it derives from as well as other types that extend the type schema. However, while it was included with WinFS, it was released as an unsupported tool. It presents a graphical interface for writing Opath queries. It can be used by selecting target object type and specifying the parameters of the query. It also includes Intellisense-like parameter completion feature. It can then be used to perform visualization tasks like binding results of a query to a DataGrid control, create views of the data in WinFS itself, or just extract the query string. Project "Orange" Microsoft launched a project to build a data visualization application for WinFS. It was codenamed "Project Orange" and was supposedly built using Windows Presentation Foundation. It was supposed to provide exploration of Items stored in WinFS stores, and data relationships were supposed to be a prominent part of the navigation model. It was supposed to let people allow organization of the WinFS stores graphically as well – productizing many of the concepts shown in the IWish Concept Video WMV File. However, since the WinFS project went dark, the status of this project is unknown. ==See also==
tickerdossier.comtickerdossier.substack.com