Overview
With so many different data and information systems aggregating on farms and ranches today, that can be either local, cloud, sensor-based tools, and platforms, how do we even try to connect them all and what does that system even look like in the future. With this varying and growing number of independent and disparate data systems and platforms from a plethora of places and locations from across the farm or ranch, it does mean we need to think about a process of hierarchical control over this sea of systems.
Without doing so, we only increase the wider cybersecurity threat, the loss of data value and limit getting to real-time business intelligence, that’s scalable and agile. Farms and ranches are unique environments for IoT and Big data technologies, similar to in-place manufacturing facilities in the desire to become efficient and effective through IoT and Big data deployments. While the application of IoT and Big data tools on farms or ranches there are very few stand-in-place locations with living organisms that are as unpredictable as humans, while having geolocation challenges, especially in North America. None the less this sector is evolving to develop and deploy tech to their unique AgriTech needs; sustainability, regenerative agriculture, welfare and a life worth living for all livestock, profitability and vibrant rural communities.
The current generation of on-farm or ranch data and data systems do tend to run in a process and cycle, understanding and mapping of those data evolutions, while automating their controls, is where the advantages of data can be exploited by the data owner and the greater deployment and development of protocols and procedures to control and use their data as a valuable asset.
In order for Precision Livestock Farming (PLF) AI and ML real-time analytics and to get there we are going to have to solve some of the latency, data ownership, scalability, storage, and integration issues with data at the farm level. Understanding farm and ranch data hygiene and automation of data cleaning, classifying in a three-stage process; screening, diagnosing, and editing, at farm level ahead of intelligence, will increase accuracy and value in PLF. The AgriTech suppliers, providers and manufacturers should begin to report and describe data-cleaning methods, error types and rates, error deletion and correction rates, and differences in outcome with and without remaining outliers, which will allow for effective integration of farm data into a central location by the data owner and allow for quality data for PLF analytics.
In classifying and cleaning the data, a “temperature” must also now be defined to it, whether it is Hot, Cold or Warm data is a classification we must consider. This is not an area that has been mentioned within PLF data assessments but needs to be understood as its important for us to know who is going to see the data, how that data will be needed within other applications, the frequency it is called up, that assists with estimating the lifetime value for storage, which aids the overall architecture for a commercial farm or ranch data ecosystem and future analytics that need to be applied to the data. Beginning with mapping where the data is coming from, data can also be classified by its “temperature” in the context of its function/purpose, as follows:
- Hot data [no data delay]: this might be data that is called frequently from the database to be used. This might be data like birth dates, cow ID or other similar data that is called up frequently for use. It is usually Mission critical data used for current reporting purposes and will constantly be used for the analytics. The only criteria are, it should be highly scalable, queries need to be fast and should retrieve data within 1s to 5s (Approx).
- Cold data [sec to hour data delay]: Is the type of data that is put into long term storage and that is accessed infrequently, but data that needs to be safely archived and backed up. This is the type of data that will be used mostly for ad-hoc reporting and data that is maintained for regulatory reasons and audit purposes, that has a low concurrency.
- Warm data [used by few, but data that needs to be pulled, usually centrally]: is often illustrated as files stored on a cloud storage gateway or file server for fast retrieval, most often at a corporate headquarters or remote office/ farms. It is not real-time, but data should be retrieved between 5s to 30s (Approx). The data should be highly scalable, less fresh data and queries are accepted to be a little slow in terms of performance.
To handle these varying temperatures of data, we need to begin to look for a farm or ranch specific digital and data system and associated infrastructure that will have the scalability for the growing needs, uniqueness, and complexities of livestock production. By looking at natural systems around us, there is a page we can take out of nature’s playbook, that we can mimic and deploy in building farm or ranch data ecosystems. Our earth’s Troposphere is the first and lowest layer of the earth’s atmosphere, it is comprised of all the clouds, vapor, mountains, and the air we breathe, it accounts for 75% of the total mass of the planetary atmosphere. Arguably, a very important layer to the cycles and processes that occur on earth, that could be our answer to our farm or ranch data ecosystem.
As the next generation of internet of things [IoT], sensors and platforms increase and proliferate on farms and ranches, so does the amount of data from across our facility increase. Copying nature and deploying a farm and ranch data ecosystem to act as our data tropospheric level, allows for the encapsulation, capture and connection of all farm or ranches data, in its different forms and temperatures, in a systematic and cyclical way, see image 1.
The necessary software and hardware components are in most cases off-the-shelf to do this, widely available and in some cases open source, with software packages that many other sectors have deployed within their digital and data infrastructure. These are Google, Microsoft, IBM, Intel, Linux, AWS and others offering tools which allow us to build your farm data ecosystem into this farm tropospheric level.
Components of a Farm or Ranch Data Ecosystem.
To make a robust, agile, and scalable farm-based data ecosystem, there are three primary layers that assist us. These include the Cloud, Edge/ Fog, Mist & sensors, that execute the following functions in our naturalistically designed data ecosystem:
The Cloud – this location tends to hold data for long term storage and a variety of more long-term needs. They can be designed in different architectural forms and platforms, that are constantly evolving. This is an off-farm location that centralizes archived data. This area needs to be robust for the growing volume and variety of data from across the farm or ranch, while giving the necessary access to the different levels and varieties of data to the correct user or member of the team at the farm level, giving the data owner the necessary oversight and control of that process or processes.
Mapping, cataloging, and logging data as it arrives to the cloud, how that architecture should work, who needs that data and what “temperature” it is, is all important knowledge that needs to be understood in your farm data ecosystem. Some business or non-time critical intelligence can run at this location, this has an impact on compute cost and energy. Finally, it’s also a secure backup location to the farm and where we can draw a copy of your data, should there be an incident, damage, or loss of systems.
The Edge & Fog – this part of our ecosystem takes farm and ranch level data from disparate platforms and sensors systems to a central location at the farm or ranch and connects it to the cloud. It can also act as a location on-farm or ranch for processing and application of local based intelligence that benefits the team, animals and business, that might be time critical or sensitive information that we don’t want or can’t do as effectively in the cloud. This is also a location where we can apply the necessary level of compute for the needs of the farm or ranch, while also thinking about its future needs as IoT and mist levels evolve and grow.
As data passes through this central on-farm or ranch edge/ fog location, it will allow for extra layers of security to be applied to the data, prior to it leaving the facility or before it heads on to its next purpose, creating a traceable footprint of your data. It is an ideal location that allows for continued and central update of firmware that is required across the whole facility, which is a huge deterrent to cybersecurity threats.
Mist & Sensors – this layer of our farm or ranch data ecosystem includes all sensors and data capturing systems and platforms that you and your farm team are using. These in-field or on-farm data tools feed up to Fog/ Edge, which then connects them to the Cloud.
The mist & sensor layer is an ever-growing segment of the ecosystem, with even more big data and smart sensors and tools becoming available [camera, wearables, sampling etc..], that give on-farm teams greater insight to key areas; welfare, sustainability, traceability, and profitability. With this growing layer we need to ensure that the data infrastructure is correct and that a value can be derived, while also itself being sustainable, scalable, and secure.
The integration of mist through Edge/ Fog and cloud allows for the benefits and value to be created by the data owner. The mist level of sensor systems is evolving into big data sensor tools and putting the necessary data ecosystem ahead of deploying more sensors and tools, will allow farmers and ranchers to grow their control of their digital and data resources and assets, while having the correct tools for their unique challenges.
Conclusion
Looking at our unique challenges within our sector and using a naturalistic approach in developing our tools for our agricultural digital revolution is what will put producers as leaders in controlling and owning their data as a resource. The Troposphere is an excellent process to mimic our farm or ranch data ecosystem on, that suitably gives control to the different levels we need, creating a cyclical system for farm or ranch data, that will allow farms and ranches to gain that diverse value.
Much like climate change which wreaks havoc if we don’t understand the balance of the change or rotation of the tropospheric layer that we have or contribute too, it can cause unknown problems and vulnerabilities. This is the same for our farm and ranch data ecosystems, requiring us to understand what is hot, cold, and warm data what we are creating, and putting out there, while identifying and creating value, while allowing us to reduce vulnerabilities.
A farm or ranch data ecosystem built this way will allow for confidence by farmers and ranchers to take control of their data ownership, security and without error or missing data and develop their digital literacy levels. It also allows for an agile and scalable infrastructure for the growing volumes of data that are coming from a growing number of sensors and platforms that are on farms or ranches, that will assist with the seven V’s of Big data: Volume, Variety, Veracity, Variability, Velocity, Visualization, and Value.
A data ecosystem that allows for the centralization of mist or sensor systems to an edge, that feeds to the cloud, gives enhanced coordination of farm or ranch data security, data ownership and business Intelligence, that ultimately helps farmers and ranchers to address proactively some of livestock agricultures main issues of sustainability, traceability, and a life worth living for all livestock.
If farms and ranches can implement intelligent data ecosystems, it will create new values and economies that contribute to the future of livestock agriculture and allow them to lead in this “digital agricultural revolution”.