AI machine learning

  • Enabling connected transformation with Apache Kafka and TensorFlow on Google Cloud Platform
    Editor’s note: Many organizations depend on real-time data streams from a fleet of remote devices, and would benefit tremendously from machine learning-derived, automated insights based on that real-time data. Founded by the team that built Apache Kafka, Confluent offers a a streaming platform to help companies easily access data as real-time streams. Today, Confluent’s Kai Waehner describes an example describing a fleet of connected vehicles, represented by Internet of Things (IoT) devices, to explain how you can leverage the open source ecosystems of Apache Kafka and TensorFlow on Google Cloud Platform and in concert with different Google machine learning (ML) services.Imagine a global automotive company with a strategic initiative for digital transformation to improve customer experience, increase revenue, and reduce risk. Here is the initial project plan:The main goal of this transformation plan is to improve existing business processes, rather than to create new services. Therefore, cutting-edge ML use cases like sentiment analysis using Recurrent Neural Networks (RNN) or object detection (e.g. for self-driving cars) using Convolutional Neural Networks (CNN) are out of scope and covered by other teams with longer-term mandates.Instead, the goal of this initiative is to analyze and act on critical business events by improving existing business processes in the short term, meaning months, not years, to achieve some quick wins with machine learning:All these business processes are already in place, and the company depends on them. Our goal is to leverage ML to improve these processes in the near term. For example, payment fraud is a consistent problem in online platforms, and our automotive company can use a variety of data sources to successfully analyze and help identify fraud in this context. In this post, we’ll explain how the company can leverage an analytic model for continuous stream processing in real-time, and use IoT infrastructure to detect payment fraud and alert them in the case of risk.Building a scalable, mission-critical, and flexible ML infrastructureBut before we can do that, let’s talk about the infrastructure needed for this project.If you’ve spent some time with TensorFlow tutorials or its most popular wrapper framework, Keras, which is typically even easier to use, you might not think that building and deploying models is all that challenging. Today, a data scientist can build an analytic model with only a few lines of Python code that run predictions on new data with very good accuracy.However, data preparation and feature engineering can consume most of a data scientist’s time. This idea may seem to contradict what you experience when you follow tutorials, because these efforts are already completed by the tutorial’s designer. Unfortunately, there is a hidden technical debt inherent in typical machine learning systems:You can read an in-depth analysis of the hidden technical debt in ML systems here.Thus, we need to ask the fundamental question that addresses how you’ll add real business value to your big data initiatives: how can you build a scalable infrastructure for your analytics models? How will you preprocess and monitor incoming data feeds? How will you deploy the models in production, on real-time data streams, at scale, and with zero downtime?Many larger technology companies faced these challenges some years before the rest of the industry. Accordingly, they have already implemented their own solutions to many of these challenges. For example, consider:Netflix’ Meson: a scalable recommendation engineUber’s Michelangelo: a platform and technology independent ML frameworkPayPal’s real time ML pipeline for fraud detectionAll of these projects use Apache Kafka as their streaming platform. This blog post explains how to solve the above described challenges for your own use cases leveraging the open source ecosystem of Apache Kafka and a number of services on Google Cloud Platform (GCP).Apache Kafka: the rise of a streaming platformYou may already be familiar with Apache Kafka, a hugely successful open source project created at LinkedIn for big data log analytics. But today, this is just one of its many use cases. Kafka evolved from a data ingestion layer to a feature-rich event streaming platform for all the use cases discussed above. These days, many enterprise data-focused projects build mission-critical applications around Kafka. As such, it has to be available and responsive, round the clock. If Kafka is down, their business processes stop working.The practicality of keeping messaging, storage, and processing in one distributed, scalable, fault-tolerant, high volume, technology-independent streaming platform is the primary reason for the global success of Apache Kafka in many large enterprises, regardless of industry. For example, LinkedIn processes over 4.5 trillion messages per day1 and Netflix handles over 6 petabytes of data on peak days2.Apache Kafka also enjoys a robust open source ecosystem. Let’s look at its components:Kafka Connect is an integration framework for connecting external sources / destinations into Kafka.Kafka Streams is a simple library that enables streaming application development within the Kafka framework. There are also additional Clients available for non-Java programming languages, including C, C++, Python, .NET, Go, and several others.The REST Proxy provides universal access to Kafka from any network connected device via HTTP.The Schema Registry is a central registry for the format of Kafka data—it guarantees that all data is in the proper format and can survive a schema evolution. As such, the Registry guarantees that the data is always consumable.KSQL is a streaming SQL engine that enables stream processing against Apache Kafka without writing source code.All these open source components build on Apache Kafka’s core messaging and storage layers, leveraging its high scalability, high volume and throughput, and failover capabilities. Then, if you need coverage for your Kafka deployment, we here at Confluent offer round-the-clock support and enterprise tooling for end-to-end monitoring, management of Kafka clusters, multi-data center replication, and more, with Confluent Cloud on GCP. This  Kafka ecosystem as a fully managed service includes a 99.95% service level agreement (SLA), guaranteed throughput and latency, and commercial support, while out-of-the-box integration with GCP services like Cloud Storage enable you to build out your scalable, mission-critical ML infrastructure.Apache Kafka’s open-source ecosystem as infrastructure for Machine LearningThe following picture shows an architecture for your ML infrastructure leveraging Confluent Cloud for data ingestion, model training, deployment, and monitoring:Now, with that background, we’re ready to build scalable, mission-critical ML infrastructure. Where do we start?Replicating IoT data from on-premises data centers to Google CloudThe first step is to ingest the data from the remote end devices. In the case of our automotive company, the data is already stored and processed in local data centers in different regions. This happens by streaming all sensor data from the cars via MQTT to local Kafka Clusters that leverage Confluent’s MQTT Proxy. This integration from devices to a local Kafka cluster typically is its own standalone project, because you need to handle IoT-specific challenges like constrained devices and unreliable networks. The integration can be implemented with different technologies, including low-level clients in C for microcontrollers, a REST Proxy for HTTP(S) communication, or an integration framework like Kafka Connect or MQTT Proxy. All of these components integrate natively with the local Kafka cluster so that you can leverage Kafka's features like high scalability, fault-tolerance and high throughput.The data from the different local clusters then needs to be replicated to a central Kafka Cluster in GCP for further processing and to train analytics models:Confluent Replicator is a tool based on Kafka Connect that replicates the data in a scalable and reliable way from any source Kafka cluster—regardless of whether it lives on premise or in the cloud—to the Confluent Cloud on GCP.GCP also offers scalable IoT infrastructure. If you want to ingest MQTT data directly into Cloud Pub/Sub from devices, you can also use GCP’s MQTT Bridge. Google provides open-source Kafka Connect connectors to get data from Cloud Pub/Sub into Kafka and Confluent Cloud so that you can make the most of KSQL with both first- and third-party logging integration.Data preprocessing with KSQLThe next step is to preprocess your data at scale. You likely want to do this in a reusable way, so that you can ingest the data into other pipelines, and to preprocess the real-time feeds for predictions in the same way once you’ve deployed the trained model.Our automotive company leverages KSQL, the open source streaming SQL engine for Apache Kafka, to do filtering, transformation, removal of personally identifiable information (PII), and feature extraction:This results in several tangible benefits:High throughput and scalability, failover, reliability, and infrastructure-independence, thanks to the core Kafka infrastructurePreprocessing data at scale with no codeUse SQL statements for interactive analysis and at-scale deployment to productionLeveraging Python using KSQL’s REST interfaceReusing preprocessed data for later deployment, even at the edge (outside of the cloud, possibly on embedded systems)Here’s what a continuous query looks like:You can then deploy this stream to one or more KSQL server instances to process all incoming sensor data in a continuous manner.Data ingestion with Kafka ConnectAfter preprocessing the data, you need to ingest it into a data store to train your models. Ideally, you should format and store in a flexible way, so that you can use it with multiple ML solutions and processes. But for today, the automotive company focuses on using TensorFlow to build neural networks that perform anomaly detection with autoencoders as a first use case. They use Cloud Storage as scalable, long-term data store for the historical data needed to train the models.In the future, the automotive company also plans to build other kinds of models using open source technologies like or for algorithms beyond neural networks. Deep Learning with TensorFlow is helpful, but it doesn’t fit every use case. In other scenarios, a random forest tree, clustering, or naïve Bayesian learning is much more appropriate due to simplicity, interpretability, or computing time.In other cases, you might be able to reduce efforts and costs a lot by using prebuilt and managed analytic models in Google’s API services like Cloud Vision for image recognition, Cloud Translate for translation between languages, or Cloud Text-to-Speech for speech synthesis. Or if you need to build custom models, Cloud AutoML might be the ideal solution to easily build out your deployment without the need for a data scientist.You can then use Kafka Connect as your ingestion layer because it provides several benefits:Kafka’s core infrastructure advantages: high throughput and scalability, fail-over, reliability, and infrastructure-independenceOut-of-the-box connectivity to various sources and sinks for different analytics and non-analytics use cases (for example, Cloud Storage, BigQuery, Elasticsearch, HDFS, MQTT)A set of out-of-the-box integration features, called Simple Message Transformation (SMT), for data (message) enrichment, format conversion, filtering, routing, and error-handlingModel training with Cloud ML Engine and TensorFlowAfter you’ve ingested your historical data into Cloud Storage, you’re now able to train your models at extreme scale using TensorFlow and TPUs on Google ML Engine. One major benefit of running your workload on a public cloud is that you can use powerful hardware in a flexible way. Spin it up for training and stop it when finished. The pay-as-you-go principle allows you to use cutting-edge hardware while still controlling your costs.In the case of our automotive company, it needs to train and deploy custom neural networks that include domain-specific knowledge and experience. Thus, they cannot use managed, pre-fabricated ML APIs or Cloud AutoML here. Cloud ML Engine provides powerful API and an easy-to-use web UI to train and evaluate different models:Although Cloud ML Engine supports other frameworks, TensorFlow is a great choice because it is open source and highly scalable, features out-of-the-box integration with GCP, offers a variety of tools (like TensorBoard for Keras), and has grown a sizable community.Replayability with Apache Kafka: a log never forgetsWith Apache Kafka as the streaming platform in your machine learning infrastructure, you can easily:Train different models on the same dataTry out different ML frameworksLeverage Cloud AutoML if and where appropriateDo A/B testing to evaluate different modelsThe architecture lets you leverage other frameworks besides TensorFlow later—if appropriate. Apache Kafka allows you to replay the data again and again over time to train different analytic models with the same dataset:In the above example, using TensorFlow, you can train multiple alternative models on historical data stored in Cloud Storage. In the future, you might want or need to use other machine learning techniques. For example, if you want to offer AutoML services to less experienced data scientists, you might train Google AutoML on Cloud Storage, or experiment with alternative, third party AutoML solutions like DataRobot or H2O Driverless, which leverage HDFS as storage on Cloud Dataproc, a managed service for Apache Hadoop and Spark.Alternative methods for model deployment and serving (inference)The automotive company is now ready to deploy its first models to do real-time predictions at scale. Two alternatives exist for model deployment:Option 1: RPC communication for model inference on your model serverCloud ML Engine allows you to deploy your trained models directly to a model server (based on TensorFlow Serving).Pros of using a model server:Simple integration with existing technologies and organizational processesEasier to understand if you come from the non-streaming (batch) worldAbility to migrate to true streaming down the roadModel management built-in for different models, versioning and A/B testingOption 2: Integrate model inference natively into your streaming applicationHere are some challenges you might encounter as you deploy your model natively in your streaming application:Worse latency: classification requires a remote call instead of local inferenceNo offline inference: on a remote or edge device, you might have limited or no connectivityCoupling the availability, scalability, and latency/throughput of your Kafka Streams application with the SLAs of the RPC interfaceOutliers or externalities (e.g., in case of failure) not covered by Kafka processingFor each use case, you have to assess the trade-offs and decide whether you want to deploy your model in a model server or natively in the application.Deployment and scalability of your client applicationsConfluent Cloud running in conjunction with GCP services ensures high availability and scalability for the machine learning infrastructure described above. You won’t need to worry about operations, just use the components to build your analytic models. However, what about the deployment and dynamic scalability of the Kafka clients, which use the analytic models to do predictions on new incoming events in real-time?You can write these clients using any programming language like (Java, Scala, .NET, Go, Python, JavaScript), Confluent REST Proxy, Kafka Streams or KSQL applications. Unlike on a Kafka server, clients need to scale dynamically to accommodate the load. Whichever option you choose for writing your Kafka clients, Kubernetes is a more and more widely adopted solution that handles deployment, dynamic scaling, and failover. Although it would be out of scope to introduce Kubernetes in this post, Google Kubernetes Engine Quickstart Guide can help you set up your own Kubernetes cluster on GCP in minutes. If you need to learn more details about the container orchestration engine itself, Kubernetes’ official website is a good starting point.The need for local data processing and model inferenceIf you’ve deployed analytics models on Google Cloud, you’ll have noticed that the service (and by extension, GCP) takes over most of the burden of deployment and operations. Unfortunately, migrating to the cloud is not always possible due to legal, compliance, security, or more technical reasons.Our automotive company is ready to use the models it built for predictions, but all the personally identifiable information (PII) data needs to be processed in its local data center. However, this demand creates a challenge, because the architecture (and some future planned integrations) would be simpler if everything were to run within one public cloud.Self-managed on-premise deployment for model serving and monitoring with KubernetesOn premises, you do not get all the advantages of GCP and Confluent Cloud—you need to operate the Apache Kafka cluster and its clients yourself.What about scaling brokers, external clients, persistent volumes, failover, and rolling upgrades? Confluent Operator takes over the challenge of operating Kafka and its ecosystem on Kubernetes, with automated provisioning, scaling, fail-over, partition rebalancing, rolling updates, and monitoring.For your clients, you face the same challenges as if you deploy in the cloud. What about dynamic load-balancing, scaling, and failover? In addition, if you use a model server on premise, you also need to manage its operations and scaling yourself.Kubernetes is an appropriate solution to solve these problems in an on-premises deployment. Using it both on-premises and on Google Cloud allows you to re-use past lessons learned and ongoing best practices.Confluent schema registry for message validation and data governanceHow can we ensure that every team in every data center gets the data they’re looking for, and that it’s consistent across the entire system?Kafka’s core infrastructure advantages: high throughput and scalability, fail-over, reliability, and infrastructure-independenceSchema definition and updatesForward- and backward-compatibilityMulti-region deploymentA mission-critical application: a payment fraud detection systemLet’s begin by reviewing the implementation of our first use case in more detail, including some code examples. We now plan to analyze historical data about payments for digital car services (perhaps for a car’s mobile entertainment system or paying for fuel at a gas station) to spot anomalies indicating possible fraudulent behavior. The model training happens in GCP, including preprocessing that anonymizes private user data. After building a good analytic model in the cloud, you can deploy it at the edge in a real-time streaming application, to analyze new transactions locally and in real time.Model training with TensorFlow on TPUsOur automotive company trained a model in Cloud ML Engine. They used Python and Keras to build an autoencoder (anomaly detection) for real-time sensor analytics, and then trained this model in TensorFlow on Cloud ML Engine leveraging Cloud TPUs (Tensor Processing Units):Google Cloud’s documentation has lots of information on how to train a model with Cloud ML Engine, including for different frameworks and use cases. If you are new to this topic, the Cloud ML Engine Getting Started guide is a good start to build your first model using TensorFlow. As a next step, you can walk through more sophisticated examples using Google ML Engine, TensorFlow and Keras for image recognition, object detection, text analysis, or a recommendation engine.The resulting, trained model is stored in Cloud Storage and thus can either deployed on a “serving” instance for live inference, or downloaded for edge deployment, as in our example above.Model deployment in KSQL streaming microservicesThere are different ways to build a real-time streaming application in the Kafka ecosystem. You can build your own application or microservice using any Kafka Client API (Java, Scala, .NET, Go, Python, node.js, or REST). Or you might want to leverage Kafka Streams (writing Java code) or KSQL (writing SQL statements)—two lightweight but powerful stream-processing frameworks that natively integrate with Apache Kafka to process high volumes of messages at scale, handle failover without data loss, and dynamically adjust scale without downtime.Here is an example of model inference in real-time. It is a continuous query that leverages the KSQL user-defined function (UDF) ‘applyFraudModel’, which embeds an autoencoder:You can deploy this KSQL statement as a microservice to one or more KSQL servers. The called model performs well and scales as needed, because it uses the same integration and processing pipeline for both training and deploying the model, leveraging Kafka Connect for real-time streaming ingestion of the sensor data, and KSQL (with Kafka Streams engine under the hood) for preprocessing and model deployment.You can build your own stateless or stateful UDFs for KSQL easily. You can find the above KSQL ML UDF and a step-by-step guide on using MQTT sensor data.If you’d prefer to leverage Kafka Streams to write your own Java application instead of writing KSQL, you might look at some code examples for deployment of a TensorFlow model in Kafka Streams.Monitoring your ML infrastructureOn GCP, you can leverage tools like Stackdriver, which allows monitoring and management for services, containers, applications, and infrastructure. Conventionally, organizations use Prometheus and JMX notifications for notifications and updates on their Kubernetes and Kafka integrations. Still, there is no silver bullet for monitoring your entire ML infrastructure, and adding an on-premises deployment creates additional challenges, since you have to set up your own monitoring tools instead of using GCP.Feel free to use the monitoring tools and frameworks you and your team already know and like. Ideally, you need to monitor your data ingestion, processing, training, deployment, accuracy, A/B testing workloads all in a scalable, reliable way. Thus, the Kafka ecosystem and GCP can provide the right foundation for your monitoring needs. Additional frameworks, services, or UIs can help your team to effectively monitor its infrastructure.Scalable, mission-critical machine learning infrastructure with GCP and Confluent CloudIn sum, we’ve shown you how you might build scalable, mission-critical, and even vendor-agnostic machine learning infrastructure. We’ve also shown you how you might leverage the open source Apache Kafka ecosystem to do data ingestion, processing, model inference, and monitoring. GCP offers all the compute power and extreme scale to run the Kafka infrastructure as a service, plus out-of-the-box integration with platform services such as Cloud Storage, Cloud ML Engine, GKE, and others. No matter where you want to deploy and run your models (in the cloud vs. on-premises; natively in-app vs. on a model server), GCP and Confluent Cloud are a great combination to set up your machine learning infrastructure.If you need coverage for your Kafka deployment, Confluent offers round-the-clock support and enterprise tooling for end-to-end monitoring, management of Kafka clusters, multi-data center replication, and more. If you’d like to learn more about Confluent Cloud, check out:The Confluent Cloud site, which provides more information about Confluent’s Enterprise and Professional offerings.The Confluent Cloud Professional getting started video.Our instructions to spin up your Kafka cloud instance on GCP here.Our community Slack group, where you can post questions in the #confluent-cloud channel.1. Read more »
  • Making AI-powered speech more accessible—now with more options, lower prices, and new languages and voices
    The ability to recognize and synthesize speech is critical for making human-machine interaction natural, easy, and commonplace, but it’s still too rare. Today we're making our Cloud Speech-to-Text and Text-to-Speech products more accessible to companies around the world, with more features, more voices (roughly doubled), more languages in more countries (up 50+%), and at lower prices (by up to 50% in some cases).Making Cloud Speech-to-Text more accessible for enterprisesWhen creating intelligent voice applications, speech recognition accuracy is critical. Even at 90% accuracy, it's hard to have a useful conversation. Unfortunately, many companies build speech applications that need to run on phone lines and that produce noisy results, and that data has historically been hard for AI-based speech technologies to interpret.For these situations with less than pristine data, we announced premium models for video and enhanced phone in beta last year, developed with customers who opted in to share usage data with us via data logging to help us refine model accuracy. We are excited to share today that the resulting enhanced phone model now has 62% fewer transcription errors (improved from 54% last year), while the video model, which is based on technology similar to what YouTube uses for automatic captioning, has 64% fewer errors. In addition, the video model also works great in settings with multiple speakers such as meetings or podcasts.The enhanced phone model was initially available only to customers participating in the opt-in data logging program announced last year. However, many large enterprises have been asking us for the option to use the enhanced model without opting into data logging. Starting today, anyone can access the enhanced phone model, and customers who choose the data logging option pay a lower rate, bringing the benefits of improved accuracy to more users.In addition to the general availability of both premium models, we’re also announcing the general availability of multi-channel recognition, which helps the Cloud Speech-to-Text API distinguish between multiple audio channels (e.g., different people in a conversation), which is very useful for doing call or meeting analytics and other use cases involving multiple participants. With general availability, all these features now qualify for an SLA and other enterprise-level guarantees.Cloud Speech-to-Text at LogMeInLogMeIn is an example of a customer that requires both accuracy and enterprise scale: Every day, millions of employees use its GoToMeeting product to attend an online meeting. Cloud Speech-to-Text lets LogMeIn automatically create transcripts for its enterprise GoToMeeting customers, enabling users to collaborate more effectively. “LogMeIn continues to be excited about our work with Google Cloud and its market-leading video and real-time speech to text technology. After an extensive market study for the best Speech-to-Text video partner, we found Google to be the highest quality and offered a useful array of related technologies. We continue to hear from our customers that the feature has been a way to add significant value by capturing in-meeting content and making it available and shareable post-meeting. Our work with Google Cloud affirms our commitment to making intelligent collaboration a fundamental part of our product offering to ultimately add more value for our global UCC customers.” - Mark Strassman, SVP and General Manager, Unified Communications and Collaboration (UCC) at LogMeIn.Making Cloud Speech-to-Text more accessible through lower pricing (up to 50% cheaper)Lowering prices is another way we are making Cloud Speech-to-Text more accessible. Starting now:For standard models and the premium video model, customers that opt-in to our data logging program will now pay 33% less for all usage that goes through the program.We’ve cut pricing for the premium video model by 25%, for a total savings of 50% for current video model customers who opt-in to data logging.Making Cloud Text-to-Speech accessible across more countriesWe’re also excited to help enterprises benefit from our research and experience in speech synthesis. Thanks to unique access to WaveNet technology powered by Google Cloud TPUs, we can build new voices and languages faster and easier than is typical in the industry: Since our update last August, we’ve made dramatic progress on Cloud Text-to-Speech, roughly doubling the number of overall voices, WaveNet voices, and WaveNet languages, and increasing the number of supported languages overall by ~50%, including:Support for seven new languages or variants, including Danish, Portuguese/Portugal, Russian, Polish, Slovakian, Ukrainian, and Norwegian Bokmål (all in beta). This update expands the list of supported languages to 21 and enables applications for millions of new end-users.31 new WaveNet voices (and 24 new standard voices) across those new languages. This gives more enterprises around the world access to our speech synthesis technology, which based on mean opinion score has already closed the quality gap with human speech by 70%. You can find the complete list of languages and voices here.20 languages and variants with WaveNet voices, up from nine last August--and up from just one a year ago when Cloud Text-to-Speech was introduced, marking a broad international expansion for WaveNet.In addition, the Cloud Text-to-Speech Device Profiles feature, which optimizes audio playback on different types of hardware, is now generally available. For example, some customers with call center applications optimize for interactive voice response (IVR), whereas others that focus on content and media (e.g., podcasts) optimize for headphones. In every case, the audio effects are customized for the hardware.Get started todayIt’s easy to give Cloud Speech products a try—check out the simple demos on the Cloud Speech-to-Text and Cloud Text-to-Speech landing pages. If you like what you see, you can use the $300 GCP credit to start testing. And as always, the first 60 minutes of audio you process every month with Cloud Speech-to-Text is free. Read more »
  • AI in depth: monitoring home appliances from power readings with ML
    As the popularity of home automation and the cost of electricity grow around the world, energy conservation has become a higher priority for many consumers. With a number of smart meter devices available for your home, you can now measure and record overall household power draw, and then with the output of a machine learning model, accurately predict  individual appliance behavior simply by analyzing meter data. For example, your electric utility provider might send you a message if it can reasonably assess that you left your refrigerator door open, or if the irrigation system suddenly came on at an odd time of day.In this post, you’ll learn how to accurately identify home appliances’ (e.g. electric kettles and washing machines, in this dataset) operating status using smart power readings, together with modern machine learning techniques such as long short-term memory (LSTM) models. Once the algorithm identifies an appliance’s operating status, we can then build out a few more applications. For example:Anomaly detection: Usually the TV is turned off when there is no one at home. An application can send a message to the user if the TV turns on at an unexpected or unusual time.Habit-improving recommendations: We can present users the usage patterns of home appliances in the neighborhood at an aggregated level so that they can compare or refer to the usage patterns and optimize the usage of their home appliances.We developed our end-to-end demo system entirely on Google Cloud Platform, including data collection through Cloud IoT Core, a machine learning model built using TensorFlow and trained on Cloud Machine Learning Engine, and real-time serving and prediction made possible by Cloud Pub/Sub, App Engine and Cloud ML Engine. As you progress through this post, you can access the full set of source files in the GitHub repository here.IntroductionThe growing popularity of IoT devices and the evolution of machine learning technologies have brought new opportunities for businesses. In this post, you’ll learn how home appliances’ (for example, an electric kettle and a washing machine) operating status (on/off) can be inferred from gross power readings collected by a smart meter, together with state-of-the-art machine learning techniques. An end-to-end demo system, developed entirely on Google Cloud Platform (as shown in Fig. 1), includes:Data collection and ingest through Cloud IoT Core and Cloud Pub/SubA machine learning model, trained using Cloud ML EngineThat same machine learning model, served using Cloud ML Engine together with App Engine as a front endData visualization and exploration using BigQuery and ColabFigure 1. System architectureThe animation below shows real-time monitoring, as real-world energy usage data is ingested through Cloud IoT Core into Colab.Figure 2. Illustration of real-time monitoringIoT extends the reach of machine learningData ingestionIn order to train any machine learning model, you need data that is both suitable and sufficient in quantity. In the field of IoT, we need to address a number of challenges in order to reliably and safely send the data collected by smart IoT devices to remote centralized servers. You’ll need to consider data security, transmission reliability, and use case-dependent timeliness, among other factors.Cloud IoT Core is a fully managed service that allows you to easily and securely connect, manage, and ingest data from millions of globally dispersed devices. The two main features of Cloud IoT Core are its device manager and its protocol bridge. The former allows you to configure and manage individual devices in a coarse-grained way by establishing and maintaining devices’ identities along with authentication after each connection. The device manager also stores each device’s logical configuration and is able to remotely control the devices—for example, changing a fleet of smart power meters’ data sampling rates. The protocol bridge provides connection endpoints with automatic load balancing for all device connections, and natively supports secure connection over industry standard protocols such as MQTT and HTTP. The protocol bridge publishes all device telemetry to Cloud Pub/Sub, which can then be consumed by downstream analytic systems. We adopted the MQTT bridge in our demo system and the following code snippet includes MQTT-specific logic.Data consumptionAfter the system publishes data to Cloud Pub/Sub, it delivers a message request to the “push endpoint,” typically the gateway service that consumes the data. In our demo system, Cloud Pub/Sub pushes data to a gateway service hosted in App Engine which then forwards the data to the machine learning model hosted in the Cloud ML Engine for inference, and at the same time stores the raw data together with received prediction results in BigQuery for later (batch) analysis.While there are numerous business-dependent use cases you can deploy based on our sample code, we illustrate raw data and prediction results visualization in our demo system. In the code repository, we have provided two notebooks:EnergyDisaggregationDemo_Client.ipynb: this notebook simulates multiple smart meters by reading in power consumption data from a real world dataset and sends the readings to the server. All Cloud IoT Core-related code resides in this notebook.EnergyDisaggregationDemo_View.ipynb: this notebook allows you to view raw power consumption data from a specified smart meter and our model's prediction results in almost real time.If you follow the deployment instructions in the README file and in the accompanying notebooks, you should be able to reproduce the results shown in Figure 2. Meanwhile, if you’d prefer to build out your disaggregation pipeline in a different manner, you can also use Cloud Dataflow and Pub/Sub I/O to build an app with similar functionality.Data processing and machine learningDataset introduction and explorationWe trained our model to predict each appliance’s on/off status from gross power readings, using the UK Domestic Appliance-Level Electricity (UK-DALE, publicly available here1) dataset  in order for this end-to-end demo system to be reproducible. UK-DALE records both whole-house power consumption and usage from each individual appliance every 6 seconds from 5 households. We demonstrate our solution using the data from house #2, for which the dataset includes a total of 18 appliances’ power consumption. Given the granularity of the dataset (a sample rate of ⅙ Hz), it is difficult to estimate appliances with relatively tiny power usage. As a result, appliances such as laptops and computer monitors are removed from this demo. Based on a data exploration study shown below, we selected eight appliances out of the original 18 items as our target appliances: a treadmill, washing machine, dishwasher, microwave, toaster, electric kettle, rice cooker and “cooker,” a.k.a., electric stovetop.The figure below shows the power consumption histograms of selected appliances. Since all the appliances are off most of the time, most of the readings are near zero. Fig. 4 shows the comparisons between aggregate power consumption of selected appliances (`app_sum`) and the whole-house power consumption (`gross`). It is worth noting that the input to our demo system is the gross consumption (the blue curve) because this is the most readily available power usage data, and is even measurable outside the home.Figure 3. Target appliances and demand histogramsFigure 4. Data sample from House #2 (on 2013-07-04 UTC)The data for House #2 spans from late February to early October 2013. We used data from June to the end of September in our demo system due to missing data at both ends of the period. The descriptive summary of selected appliances is illustrated in Table 1. As expected, the data is extremely imbalanced in terms of both “on” vs. “off” for each appliance and power consumption scale of each appliance, which introduces the main difficulty of our prediction task.Table 1. Descriptive summary of power consumptionPreprocessing the dataSince UK-DALE did not record individual appliance on/off status, one key preprocessing step is to label the on/off status of each appliance at each timestamp. We assume an appliance to be “on” if its power consumption is larger than one standard deviation from the sample mean of its power readings, given the fact that appliances are off most of the time and hence most of the readings are near zero. The code for data preprocessing can be found in the notebook provided, and you can also download the processed data from here.With the preprocessed data in CSV format, TensorFlow’s Dataset class serves as a convenient tool for data loading and transformation—for example, the input pipeline for machine learning model training. For example, in the following code snippet lines 7 - 9 load data from the specified CSV file and lines 11 - 13 transform data into our desired time-series sequence.In order to address the data imbalance issue, you can either down-sample the majority class or up-sample the minority class. In our case, we propose a probabilistic negative down-sampling method: we’ll preserve the subsequences in which at least one appliance remains on, but we’ll filter the subsequences with all appliances off, based on a certain probability and threshold. The filtering logic integrates easily with the API, as in the following code snippet:Finally, you’ll want to follow best practices from Input Pipeline Performance Guide to ensure that your GPU or TPU (if they are used to speed up training process) resources are not wasted while waiting for the data to load from the input pipeline. To maximize usage, we employ parallel mapping to parallelize data transformation and prefetch data to overlap the preprocessing and model execution of a training step, as shown in the following code snippet:The machine learning modelWe adopt a long short-term memory (LSTM) based network as our classification model. Please see Understanding LSTM Networks for an introduction to recurrent neural networks and LSTMs. Fig. 5 depicts our model design, in which an input sequence of length n is fed into a multilayered LSTM network, and prediction is made for all m appliances. A dropout layer is added for the input of LSTM cell, and the output of the whole sequence is fed into a fully connected layer. We implemented this model as a TensorFlow estimator.Figure 5. LSTM based model architectureThere are two ways of implementing the above architecture: TensorFlow native API (tf.layers and tf.nn) and Keras API (tf.keras). Compared to TensorFlow’s native API, Keras serves as a higher level API that lets you train and serve deep learning models with three key advantages: ease of use, modularity, and extensibility. tf.keras is TensorFlow's implementation of the Keras API specification. In the following code sample, we implemented the same LSTM-based classification model using both methods, so that you can compare the two:Model authoring using TensorFlow’s native API:Model authoring using the Keras API:Training and hyperparameter tuningCloud Machine Learning Engine supports both training and hyperparameter tuning. Figure 6 shows the average (over all appliances) precision, recall and f_score for multiple trials with different combinations of hyperparameters. We observed that hyperparameter tuning significantly improves model performance.Figure 6. Learning curves from hyperparameter tuning.We selected two experiments with optimal scores from hyperparameter tunings and report their performances in Table 2.Table 2. Hyper-parameter tuning of selected experimentsTable 3 lists the precision and recall of each individual appliance. As mentioned in the previous “Dataset introduction and exploration” section, the cooker and the treadmill (“running machine”) are difficult to predict, because their peak power consumptions were significantly lower than other appliances.Table 3. Precision and recall of predictions for individual appliancesConclusionWe have provided an end-to-end demonstration of how you can use machine learning to determine the operating status of home appliances accurately, based on only smart power readings. Several products including Cloud IoT Core,  Cloud Pub/Sub,  Cloud ML Engine, App Engine and BigQuery are orchestrated to support the whole system, in which each product solves a specific problem required to implement this demo, such as data collection/ingestion, machine learning model training, real time serving/prediction, etc. Both our code and data are available for those of you who would like to try out the system for yourself.We are optimistic that both we and our customers will develop ever more interesting applications at the intersection of more capable IoT devices and fast-evolving machine learning algorithms. Google Cloud provides both the IoT infrastructure and machine learning training and serving capabilities that make newly capable smart IoT deployments both a possibility and a reality.1. Jack Kelly and William Knottenbelt. The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes. Scientific Data 2, Article number:150007, 2015, DOI:10.1038/sdata.2015.7. Read more »
  • How Box Skills can optimize your workflow with the help of Cloud AI
    Have you ever had to manually upload and tag a lot of files? It’s no fun. Increasingly though, machine learning algorithms can help you or your team classify and tag large volumes of content automatically. And if your company uses Box, a popular file sharing, storage and collaboration service, you can now apply Google ML services to your files with just a few lines of code, using the Box Skills Kit, a new framework within Box’s developer toolkit.With technologies like image recognition, speech-to-text transcription, and natural language understanding, Google Cloud makes it easy to enrich your Box files with useful metadata. For example, if you have lots of images in your repository, you can use the Cloud Vision API to understand more about the image, such as objects or landmarks in an image, or in documents, —or even parse their contents and identify elements that determine the document’s category. If your needs extend beyond functionality provided by Cloud Vision, you can point your Skill at a custom endpoint that serves your own custom-trained model.An example integration in actionNow, let’s look at an example. Many businesses use Box to store images of their products. With the Box Skills Kit and the product search functionality in the Cloud Vision API, you can automatically catalog these products. When a user uploads a new product image into Box, the product search feature within the Vision API helps identify similar products in the catalog, as well as the maximum price for such a product.Configuring and deploying a product search Box skillLet’s look at how you can use the Box Skills Kit to implement the use case outlined above.1.Create an endpoint for your Skill   a. Follow this QuickStart guide.   b. You can use this API endpoint to call a pre-trained machine learning model to classify new data.   c. Create a Cloud Function to point your Box Skill at the API endpoint created above.   d. Clone the following repository.   e. Next, follow the instructions to deploy the function to your project.   f. Make a note of the endpoint’s URI.2.Configure a Box Custom Skills App in Box, then configure it to point to the Cloud Function created above.   a. Follow the instructions.   b. Then these instructions.And there you have it. You now have a new custom Box Skill enabled by Cloud AI that’s ready to use. Try uploading a new image to your Box drive and notice that maximum retail price and information on similar products are both displayed under the “skills” console.Using your new SkillNow that you’re all set up, you can begin by uploading an image file of household goods, apparel, or toys into your Box drive. The upload triggers a Box Skill event workflow, which calls a Cloud Function you deployed in Google Cloud and whose endpoint you will specify in the Box Admin Console. The Cloud Function you create then uses the Box Skills kit's FileReader API to read the base64-encoded image string, automatically sent by Box when the upload trigger occurs. The Function then calls the product search function of Cloud Vision, and creates a Topics Card with data returned from the product search function. Next, it creates a Faces card in which to populate a thumbnail that it scaled from the original image. Finally, the function persists the skills card within Box using the skillswriter API. Now, you can open the image in Box drive and click on the "skills" menu (which expands, when you click the “magic wand” icon on the right), and you’ll see product catalog information, with similar products and maximum price populated.What’s next?Over the past several years, Google Cloud and Box have built a variety of tools to make end users more productive. Today, the Box Skills integration opens the door to a whole new world of advanced AI tools and services: in addition to accessing pre-trained models via the Vision API, Video Intelligence API or Speech-to-Text API, data scientists can train and host custom models written in TensorFlow, sci-kit learn, Keras, or PyTorch on Cloud ML Engine. Lastly, Cloud AutoML lets you train a model on your dataset without having to write any code. Whatever your levels of comfort with code or data science, we’re committed to making it easy for you to run machine learning-enhanced annotations on your data.You can find all the code discussed in this post and its associated documentation in its GitHub repository. Goodbye, tedious repetition! Hello, productivity. Read more »
  • Spam does not bring us joy—ridding Gmail of 100 million more spam messages with TensorFlow
    1.5 billion people use Gmail every month, and 5 million paying businesses use Gmail in the workplace as a part of G Suite. For consumers and businesses alike, a big part of Gmail’s draw is its built-in security protections.Good security means constantly staying ahead of threats, and our existing ML models are highly effective at doing this—in conjunction with our other protections, they help block more than 99.9 percent of spam, phishing, and malware from reaching Gmail inboxes. Just as we evolve our security protections, we also look to advance our machine learning capabilities to protect you even better.That’s why we recently implemented new protections powered by TensorFlow, an open-source machine learning (ML) framework developed at Google. These new protections complement existing ML and rules-based protections, and they’ve successfully improved our detection capabilities. With TensorFlow, we are now blocking around 100 million additional spam messages every day.Where did we find these 100 million extra spam messages? We’re now blocking spam categories that used to be very hard to detect. Using TensorFlow has helped us block image-based messages, emails with hidden embedded content, and messages from newly created domains that try to hide a low volume of spammy messages within legitimate traffic.Given we’re already blocking the majority of spammy emails in Gmail, blocking millions more with precision is a feat. TensorFlow helps us catch the spammers who slip through that less than 0.1 percent, without accidentally blocking messages that are important to users. One person’s spam is another person’s treasureML makes catching spam possible by helping us identify patterns in large data sets that humans who create the rules might not catch; it makes it easy for us to adapt quickly to ever-changing spam attempts.ML-based protections help us make granular decisions based on many different factors. Consider that every email has thousands of potential signals. Just because some of an email’s characteristics match up to those commonly considered “spammy,” doesn’t necessarily mean it’s spam. ML allows us to look at all of these signals together to make a determination.Finally, it also helps us personalize our spam protections to each user—what one person considers spam another person might consider an important message (think newsletter subscriptions or regular email notifications from an application).Using TensorFlow to power MLBy complementing our existing ML models with TensorFlow, we’re able to refine these models even further, while allowing the team to focus less on the underlying ML framework, and more on solving the problem: ridding your inbox of spam!Applying ML at scale can be complex and time consuming. TensorFlow includes many tools that make the ML process easier and more efficient, accelerating the speed at which we can iterate. As an example, TensorBoard allows us to both comprehensively monitor our model training pipelines and quickly evaluate new models to determine how useful we expect them to be.TensorFlow also gives us the flexibility to easily train and experiment with different models in parallel to develop the most effective approach, instead of running one experiment at a time.As an open standard, TensorFlow is used by teams and researchers all over the world (There have been 71,000 forks of the public code and other open-source contributions!). This strong community support means new research and ideas can be applied quickly. And, it means we can collaborate with other teams within Google more quickly and easily to best protect our users.All in all, these benefits allow us to scale our ML efforts, requiring fewer engineers to run more experiments and protect users more effectively.This is just one example of how we’re using machine learning to keep users and businesses safe, and just one application of TensorFlow. Even within Gmail, we’re currently experimenting with TensorFlow in other security-related areas, such as phishing and malware detection, as part of our continuous efforts to keep users safe.And you can use it, too. Google open-sourced TensorFlow in 2015 to make ML accessible for everyone—so that many different organizations can take advantage of the technology that powers critical capabilities like spam prevention in Gmail and more.To learn more about TensorFlow and the companies that are using it, visit To learn more about the security benefits of G Suite, download this eBook. Read more »
  • Making the machine: the machine learning lifecycle
    As a Googler, one of my roles is to educate the software development community on machine learning (ML). The first introduction for many individuals is what is referred to as the ‘model’. While building models, tuning them, and evaluating their predictive abilities has generated a great deal of interest and excitement, many organizations still find themselves asking more basic questions, like how does machine learning fit into their software development lifecycle?In this post, I explain how machine learning (ML) maps to and fits in with the traditional software development lifecycle. I refer to this mapping as the machine learning lifecycle. This will help you as you think about how to incorporate machine learning, including models, into your software development processes. The machine learning lifecycle consists of three major phases: Planning (red), Data Engineering (blue) and Modeling (yellow).PlanningIn contrast to a static algorithm coded by a software developer, an ML model is an algorithm that is learned and dynamically updated. You can think of a software application as an amalgamation of algorithms, defined by design patterns and coded by software engineers, that perform planned tasks. Once an application is released to production, it may not perform as planned, prompting developers to rethink, redesign, and rewrite it (continuous integration/continuous delivery).We are entering an era of replacing some of these static algorithms with ML models, which are essentially dynamic algorithms. This dynamism presents a host of new challenges for planners, who work in conjunction with product owners and quality assurance (QA) teams.For example, how should the QA team test and report metrics? ML models are often expressed as confidence scores. Let’s suppose that a model shows that it is 97% accurate on an evaluation data set. Does it pass the quality test? If we built a calculator using static algorithms and it got the answer right 97% of the time, we would want to know about the 3% of the time it does not.Similarly, how does a daily standup work with machine learning models? It’s not like the training process is going to give a quick update each morning on what it learned yesterday and what it anticipates learning today. It’s more likely your team will be giving updates on data gathering/cleaning and hyperparameter tuning.When the application is released and supported, one usually develops policies to address user issues. But with continuous learning and reinforcement learning, the model is learning the policy. What policy do we want it to learn? For example, you may want it to observe and detect user friction in navigating the user interface and learn to adapt the interface (Auto A/B) to reduce the friction.Within an effective ML lifecycle, planning needs to be embedded in all stages to start answering these questions specific to your organization.Data engineeringData engineering is where the majority of the development budget is spent—as much as 70% to 80% of engineering funds in some organizations. Learning is dependent on data—lots of data, and the right data. It’s like the old software engineering adage: garbage in, garbage out. The same is true for modeling: if bad data goes in, what the model learns is noise.In addition to software engineers and data scientists, you really need a data engineering organization. These skilled engineers will handle data collection (e.g., billions of records), data extraction (e.g., SQL, Hadoop), data transformation, data storage and data serving. It’s the data that consumes the vast majority of your physical resources (persistent storage and compute). Typically due to the magnitude in scale, these are now handled using cloud services versus traditional on-prem methods.Effective deployment and management of data cloud operations are handled by those skilled in data operations (DataOps). The data collection and serving are handled by those skilled in data warehousing (DBAs). The data extraction and transformation are handled by those skilled in data engineering (Data Engineers), and data analysis are handled by those skilled in statistical analysis and visualization (Data Analysts).ModelingModeling is integrated throughout the software development lifecycle. You don’t just train a model once and you’re done. The concept of one-shot training, while appealing in budget terms and simplification, is only effective in academic and single-task use cases.Until fairly recently, modeling was the domain of data scientists. The initial ML frameworks (like Theano and Caffe) were designed for data scientists. ML frameworks are evolving and today are more in the realm of software engineers (like Keras and PyTorch). Data scientists play an important role in researching the classes of machine learning algorithms and their amalgamation, advising on business policy and direction, and moving into roles of leading data driven teams.But as ML frameworks and AI as a Service (AIaaS) evolve, the majority of modeling will be performed by software engineers. The same goes for feature engineering, a task performed by today’s data engineers: with its similarities to conventional tasks related to data ontologies, namespaces, self-defining schemas, and contracts between interfaces, it too will move into the realm of software engineering. In addition, many organizations will move model building and training to cloud-based services used by software engineers and managed by data operations. Then, as AIaaS evolves further, modeling will transition to a combination of turnkey solutions accessible via cloud APIs, such as for Cloud Vision and Cloud Speech-to-Text, and customizing pre-trained algorithms using transfer learning tools such as AutoML.Frameworks like Keras and PyTorch have already transitioned away symbol programming into imperative programming (the dominant form in software development), and incorporate object-oriented programming (OOP) principles such as inheritance, encapsulation, and polymorphism. One should anticipate that other ML frameworks will evolve to include object relational models (ORM), which we already use for databases, to data sources and inference (prediction). Common best practices will evolve and industry-wide design patterns will become defined and published, much like how Design Patterns by the Gang of Four influenced the evolution of OOP.Like continuous integration and delivery, continuous learning will also move into build processes, and be managed by build and reliability engineers. Then, once your application is released, its usage and adaptation in the wild will provide new insights in the form of data, which will be fed back to the modeling process so the model can continue learning.As you can see, adopting machine learning isn’t simply a question of learning to train a model, and you’re done. You need to think deeply about how those ML models will fit into your existing systems and processes, and grow your staff accordingly. I, and all the staff here at Google, wish you the best in your machine learning journey, as you upgrade your software development lifecycle to accommodate machine learning. To learn more about machine learning on Google Cloud here, visit our Cloud AI products page. Read more »
  • Exoplanets, astrobiological research, and Google Cloud: What we learned from NASA FDL’s Reddit AMA
    Are we alone in the universe? Does intelligent life exist on other planets? If you’ve ever wondered about these things, you’re not the only one. Last summer, we partnered with NASA's Frontier Development Lab (FDL) to help find answers to these questions—you can read about some of this work in this blog post. And as part of this work we partnered with FDL researchers to host an AMA (“ask me anything”) to answer all those burning questions from Redditlings far and wide. Here are some of the highlights:Question: What can AI do to detect intelligent life on other planets?Massimo Mascaro, Google Cloud Director of Applied AI: AI can help extract the maximum information from the very faint and noisy signals we can get from our best instruments. AI is really good at detecting anomalies and in digging through large amounts of data and that's pretty much what we do when we search for life in space.Question: About how much data is expected to be generated during this mission? Are we looking at the terabyte, 10s of terabytes, or 100s of terabytes of data?Megan Ansdell, Planetary Scientist with a specialty in exoplanets: The TESS mission will download ~6 TB of data every month as it observes a new sector of sky containing 16,000 target stars at 2-minute cadence. The mission lifetime is at least 2 years, which means TESS will produce on the order of 150 TB of data. You can learn more about the open source deep learning models that have been developed to sort through the data here.Question: What does it mean to simulate atmospheres?Giada Arney, Astronomy and astrobiology (mentor): Simulating atmospheres for me involves running computer models where I provide inputs to the computer on gases in the atmosphere, “boundary conditions”, temperature and more. These atmospheres can then be used to simulate telescopic observations of similar exoplanets so that we can predict what atmospheric features might be observable with future observatories for different types of atmospheres.Question: How useful is a simulated exoplanet database?Massimo Mascaro: It's important to have a way to simulate the variability of the data you could observe, before observing it, to understand your ability to distinguish patterns, to plan on how to build and operate instruments and even to plan how to analyze the data eventually.Giada Arney: Having a database of different types of simulated worlds will allow us to predict what types of properties we’ll be able to observe on a diverse suite of planets. Knowing these properties will then help us to think about the technological requirements of future exoplanet observing telescopes, allowing us to anticipate the unexpected!Question: Which off-the-shelf Google Cloud AI/ML APIs are you using?Massimo Mascaro, Google Cloud Director of Applied AI: We've leveraged a lot of Google Cloud’s infrastructure, in particular Compute Engine and GKE, to both experiment with data and to run computation on large scale (using up to 2500 machines simultaneously), as well as TensorFlow and PyTorch running on Google Cloud to train deep learning models for the exoplanets and astrobiology experiments.Question: What advancements in science can become useful in the future other than AI?Massimo Mascaro: AI is just one of the techniques science can benefit in our times. I would put in that league definitely the wide access to computation. This is not only helping science in data analysis and AI, but in simulation, instrument design, communication, etc.Question: What do you think are the key things that will inspire the next generation of astrophysicists, astrobiologists, and data scientists?Sara Jennings, Deputy Director, NASA FDL: For future data scientists, I think it will be the cool problems like the ones we tackle at NASA FDL, which they will be able to solve using new and ever increasing data and techniques. With new instruments and data analysis techniques getting so much better, we're now at a moment where asking question such as whether there's life outside our planet is not anymore preposterous, but real scientific work.Daniel Angerhausen, Astrophysicist with expertise spanning astrobiology to exoplanets (mentor): I think one really important point is that we see more and more women in science. This will be such a great inspiration for girls to pursue careers in STEM. For most of the history of science we were just using 50 percent of our potential and this will hopefully be changed by our generation.You can read the full AMA transcript here. Read more »
  • AI in Depth: Cloud Dataproc meets TensorFlow on YARN: Let TonY help you train right in your cluster
    Apache Hadoop has become an established and long-running framework for distributed storage and data processing. Google’s Cloud Dataproc is a fast, easy-to-use, fully managed cloud service for running Apache Spark and Apache Hadoop clusters in a simple, cost-efficient way. With Cloud Dataproc, you can set up a distributed storage platform without worrying about the underlying infrastructure. But what if you want to train TensorFlow workloads directly on your distributed data store?This post will explain how to install a Hadoop cluster for LinkedIn open-source project TonY (TensorFlow on YARN). You will deploy a Hadoop cluster using Cloud Dataproc and TonY to launch a distributed machine learning job. We’ll explore how you can use two of the most popular machine learning frameworks: TensorFlow and PyTorch.TensorFlow supports distributed training, allowing portions of the model’s graph to be computed on different nodes. This distributed property can be used to split up computation to run on multiple servers in parallel. Orchestrating distributed TensorFlow is not a trivial task and not something that all data scientists and machine learning engineers have the expertise, or desire, to do—particularly since it must be done manually. TonY provides a flexible and sustainable way to bridge the gap between the analytics powers of distributed TensorFlow and the scaling powers of Hadoop. With TonY, you no longer need to configure your cluster specification manually, a task that can be tedious, especially for large clusters.The components of our system:First, Apache HadoopApache Hadoop is an open source software platform for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware.  Hadoop services provides for data storage, data processing, data access, data governance, security, and operations.Next, Cloud DataprocCloud Dataproc is a managed Spark and Hadoop service that lets you take advantage of open source data tools for batch processing, querying, streaming, and machine learning. Cloud Dataproc’s automation capability helps you create clusters quickly, manage them easily, and save money by turning clusters off when you don't need them. With less time and money spent on administration, you can focus on your jobs and your data.And now TonYTonY is a framework that enables you to natively run deep learning jobs on Apache Hadoop. It currently supports TensorFlow and PyTorch. TonY enables running either single node or distributed training as a Hadoop application. This native connector, together with other TonY features, runs machine learning jobs reliably and flexibly.InstallationSetup a Google Cloud Platform projectGet started on Google Cloud Platform (GCP) by creating a new project, using the instructions found here.Create a Cloud Storage bucketThen create a Cloud Storage bucket. Reference here.Create a Hadoop cluster via Cloud Dataproc using initialization actionsYou can create your Hadoop cluster directly from Cloud Console or via an appropriate `gcloud` command. The following command initializes a cluster that consists of 1 master and 2 workers:When creating a Cloud Dataproc cluster, you can specify in your TonY initialization actions script that Cloud Dataproc should run on all nodes in your Cloud Dataproc cluster immediately after the cluster is set up.Note: Use Cloud Dataproc version 1.3-deb9, which is supported for this deployment. Cloud Dataproc version 1.3-deb9 provides Hadoop version 2.9.0. Check this version list for details.Once your cluster is created. You can verify that under Cloud Console > Big Data > Cloud Dataproc > Clusters, that cluster installation is completed and your cluster’s status is Running.Go to  Cloud Console > Big Data > Cloud Dataproc > Clusters and select your new cluster:You will see the Master and Worker nodes.Connect to your Cloud Dataproc master server via SSHClick on SSH and connect remotely to Master server.Verify that your YARN nodes are activeExampleInstalling TonYTonY’s Cloud Dataproc initialization action will do the following:Install and build TonY from GitHub repository.Create a sample folder containing TonY examples, for the following frameworks:TensorFlowPyTorchThe following folders are created:TonY install folder (TONY_INSTALL_FOLDER) is located by default in:TonY samples folder (TONY_SAMPLES_FOLDER) is located by default in:The Tony samples folder will provide 2 examples to run distributed machine learning jobs using:TensorFlow MNIST examplePyTorch MNIST exampleRunning a TensorFlow distributed jobLaunch a TensorFlow training jobYou will be launching the Dataproc job using a `gcloud` command.The following folder structure was created during installation in `TONY_SAMPLES_FOLDER`, where you will find a sample Python script to run the distributed TensorFlow job.This is a basic MNIST model, but it serves as a good example of using TonY with distributed TensorFlow. This MNIST example uses “data parallelism,” by which you use the same model in every device, using different training samples to train the model in each device. There are many ways to specify this structure in TensorFlow, but in this case, we use “between-graph replication” using tf.train.replica_device_setter.DependenciesTensorFlow version 1.9Note: If you require a more recent TensorFlow and TensorBoard version, take a look at the progress of this issue to be able to upgrade to latest TensorFlow version.Connect to Cloud ShellOpen Cloud Shell via the console UI:Use the following gcloud command to create a new job. Once launched, you can monitor the job. (See the section below on where to find the job monitoring dashboard in Cloud Console.)Running a PyTorch distributed jobLaunch your PyTorch training jobFor PyTorch as well, you can launch your Cloud Dataproc job using gcloud command.The following folder structure was created during installation in the TONY_SAMPLES_FOLDER, where you will find an available sample script to run the TensorFlow distributed job:DependenciesPyTorch version 0.4Torch Vision 0.2.1Launch a PyTorch training jobVerify your job is running successfullyYou can track Job status from the Dataproc Jobs tab: navigate to Cloud Console > Big Data > Dataproc > Jobs.Access your Hadoop UILogging via web to Cloud Dataproc’s master node via web: http://<Node_IP>:8088 and track Job status. Please take a look at this section to see how to access the Cloud Dataproc UI.Cleanup resourcesDelete your Cloud Dataproc clusterConclusionDeploying TensorFlow on YARN enables you to train models straight from your data infrastructure that lives in HDFS and Cloud Storage. If you’d like to learn more about some of the related topics mentioned in this post, feel free to check out the following documentation links:Machine Learning with TensorFlow on GCPHyperparameter tuning on GCPHow to train ML models using GCPAcknowledgements: Anthony Hsu, LinkedIn Software Engineer; and Zhe Zhang, LinkedIn Core Big Data Infra team manager. Read more »
  • How we built a derivatives exchange with BigQuery ML for Google Next ‘18
    Financial institutions have a natural desire to predict the volume, volatility, value or other parameters of financial instruments or their derivatives, to manage positions and mitigate risk more effectively. They also have a rich set of business problems (and correspondingly large datasets) to which it’s practical to apply machine learning techniques.Typically, though, in order to start using ML, financial institutions must first hire data scientist talent with ML expertise—a skill set for which recruiting competition is high. In many cases, an organization has to undertake the challenge and expense of bootstrapping an entire data science practice. This summer, we announced BigQuery ML, a set of machine learning extensions on top of our scalable data warehouse and analytics platform. BigQuery ML effectively democratizes ML by exposing it via the familiar interface of SQL—thereby letting financial institutions accelerate their productivity and maximize existing talent pools.As we got ready for Google Cloud Next London last summer, we decided to build a demo to showcases BigQuery ML’s potential for the financial services community. In this blog post, we’ll walk through how we designed the system, selected our time-series data, built an architecture to analyze six months of historical data, and quickly trained a model to outperform a 'random guess' benchmark—all while making predictions in close to real time.Meet the Derivatives ExchangeA team of Google Cloud solution architects and customer engineers built the Derivatives Exchange in the form of an interactive game, in which you can opt to either rely on luck, or use predictions from a model running in BigQuery ML, in order to decide which options contracts will expire in-the-money. Instead of using the value of financial instruments as the “underlying” for the options contracts, we used the volume of Twitter posts (tweets) for a particular hashtag within a specific timeframe. Our goal was to show the ease with which you can deploy machine learning models on Google Cloud to predict an instrument’s volume, volatility, or value.The Exchange demo, as seen at Google Next ‘18 LondonOur primary goal was to translate an existing and complex trading prediction process into a simple illustration to which users from a variety of industries can relate. Thus, we decided to:Use the very same Google Cloud products that our customers use daily.Present a time-series that is familiar to everyone—in this case, the number of hashtag Tweets observed in a 10-minute window as the “underlying” for our derivative contracts.Build a fun, educational, and inclusive experience.When designing the contract terms, we used this Twitter time-series data in a manner similar to the strike levels specified in weather derivatives.Architectural decisionsSolution architecture diagram: the social media options marketWe imagined the exchange as a retail trading pit where, using mobile handsets, participants purchase European binary range call option contracts across various social media single names (what most people would think of as hashtags). Contracts are issued every ten minutes and expire after ten minutes. At expiry, the count of accumulated #hashtag mentions for the preceding window is used to determine which participants were holding in-the-money contracts, and their account balances are updated accordingly. Premiums are collected upon opening interest in a contract, and are refunded if the contract strikes in-the-money. All contracts pay out 1:1.We chose the following Google Cloud products to implement the demo:Compute Engine served as our job server:The implementation executes periodic tasks for issuing, expiring, and settling contracts. The design also requires a singleton process to run as a daemon to continually ingest tweets into BigQuery. We decided to consolidate these compute tasks into an ephemeral virtual machine on Compute Engine. The job server tasks were authored with node.js and shell scripts, using cron jobs for scheduling, and configured by an instance template with embedded VM startup scripts, for flexibility of deployment. The job server does not interact with any traders on the system, but populates the “market operational database” with both participant and contract status.Cloud Firestore served as our market operational database:Cloud Firestore is a document-oriented database that we use to store information on market sessions. It serves as a natural destination for the tweet count and open interest data displayed by the UI, and enables seamless integration with the front end.Firebase and App Engine provided our mobile and web applications:Using the Firebase SDK for both our mobile and web applications’ interfaces enabled us to maintain a streamlined codebase for the front end. Some UI components (such as the leaderboard and market status) need continual updates to reflect changes in the source data (like when a participant’s interest in a contract expires in-the-money). The Firebase SDK provides concise abstractions for developers and enables front-end components to be bound to Cloud Firestore documents, and therefore to update automatically whenever the source data changes.Choosing App Engine to host the front-end application allowed us to focus on UI development without the distractions of server management or configuration deployment. This helped the team rapidly produce an engaging front end.Cloud Functions ran our backend API services:The UI needs to save trades to Cloud Firestore, and Cloud Functions facilitate this serverlessly. This serverless backend means we can focus on development logic, rather than server configuration or schema definitions, thereby significantly reducing the length of our development iterations.BigQuery and BigQuery ML stored and analyzed tweetsBigQuery solves so many diverse problems that it can be easy to forget how many aspects of this project it enables. First, it reliably ingests and stores volumes of streaming Twitter data at scale and economically, with minimal integration effort. The daemon process code for ingesting tweets consists of 83 lines of Javascript, with only 19 of those lines pertaining to BigQuery.Next, it lets us extract features and labels from the ingested data, using standard SQL syntax. Most importantly, it brings ML capabilities to the data itself with BigQuery ML, allowing us to train a model on features extracted from the data, ultimately exposing predictions at runtime by querying the model with standard SQL.BigQuery ML can help solve two significant problems that the financial services community faces daily. First, it brings predictive modeling capabilities to the data, sparing the cost, time and regulatory risk associated with migrating sensitive data to external predictive models. Second, it allows these models to be developed using common SQL syntax, empowering data analysts to make predictions and develop statistical insights. At Next ‘18 London, one attendee in the pit observed that the tool fills an important gap between data analysts, who might have deep familiarity with their particular domain’s data but less familiarity with statistics; and data scientists, who possess expertise around machine learning but may be unfamiliar with the particular problem domain. We believe BigQuery ML helps address a significant talent shortage in financial services by blending these two distinct roles into one.Structuring and modeling the dataOur model training approach is as follows:First, persist raw data in the simplest form possible: filter the Twitter Enterprise API feed for tweets containing specific hashtags (pulled from a pre-defined subset), and persist a two-column time-series consisting of the specific hashtag as well as the timestamp of that tweet as it was observed in the Twitter feed.Second, define a view in SQL that sits atop the main time-series table and extracts features from the raw Twitter data. We selected features that allow the model to predict the number of tweet occurrences for a given hashtag within the next 10-minute period. Specifically:#Hashtag#fintech may have behaviors distinct from #blockchain and distinct from #brexit, so the model should be aware of this as a feature.Day of weekSunday’s tweet behaviors will be different from Thursday’s tweet behaviors.Specific intra-day windowWe sliced a 24-hour day into 144 10-minute segments, so the model can inform us on trend differences between various parts of the 24-hour cycle.Average tweet count from the past hourThese values are calculated by the view based upon the primary time-series data.Average tweet velocity from the past hourTo predict future tweet counts accurately, the model should know how active the hashtag has been in the prior hour, and whether that activity was smooth (say, 100 tweets consistently for each of the last six 10-minute windows) or bursty (say, five 10-minute windows with 0 tweets followed by one window with 600 tweets).Tweet count rangeThis is our label, the final output value that the model will predict. The contract issuance process running on the job server contains logic for issuing options contracts with strike ranges for each hashtag and 10-minute window (Range 1: 0-100, Range 2: 101-250, etc.) We took the large historical Twitter dataset and, using the same logic, stamped each example with a label indicating the range that would have been in-the-money. Just as equity option chains issued on a stock are informed by the specific stock’s price history, our exchange’s option chains are informed by the underlying hashtag's volume history.Train the model on this SQL view. BigQuery ML makes model training an incredibly accessible exercise. While remaining inside the data warehouse, we use a SQL statement to declare that we want to create a model trained on a particular view containing the source data, and using a particular column as a label.Finally, deploy the trained model in production. Again using SQL, simply query the model based on certain input parameters, just as you would query any table.Trading options contractsTo make the experience engaging, we wanted to recreate a bit of the open-outcry pit experience by having multiple large “market data” screens for attendees (the trading crowd) to track contract and participant performance. Demo participants used Pixel 2 handsets in the pit to place orders using a simple UI, from which they could allocate their credits to any or all of the three hashtags. When placing their order, they chose between relying on their own forecast, or using the predictions of a BigQuery ML model for their specific options portfolio, among the list of contracts currently trading in the market. Once the trades were made for their particular contracts, they monitored how their trades performed compared to other “traders” in real-time, then saw how accurate the respective predictions were when the trading window closed at expiration time (every 10 minutes).ML training processIn order to easily generate useful predictions about tweet volumes, we use a three-part process, First, we store tweet time series data to a BigQuery table. Second, we layer views are layered on top of this table to extract the features and labels required for model training. Finally, we use BigQuery ML to train and get predictions from the model.The canonical list of hashtags to be counted is stored within a BigQuery table named “hashtags”. This is joined with the “tweets” table to determine aggregates for each time window.Example 1: Schema definition for the “hashtags” table1. Store tweet time series data The tweet listener writes tags, timestamps, and other metadata to a BigQuery table named “tweets” that possesses the schema listed in example 2:Example 2: Schema definition for the “tweets” table2. Extract features via layered viewsThe lowest-level view calculates the count of each hashtag’s occurrence, per intraday window. The mid-level view extracts the features mentioned in the above section (“Structuring and modeling the data”). The top-level view then extracts the label (i.e., the “would-have-been in-the-money” strike range) from that time-series data. Lowest-level view The lowest-level view is defined by the SQL in example 3. The view definition contains logic to aggregate tweet history into 10-minute buckets (with 144 of these buckets per 24-hour day) by hashtag.Example 3: low-level view definitionb. Intermediate viewThe selection of some features (for example: hashtag, day-of-week or specific intraday window) is straightforward, while others (such as average tweet count and velocity for the past hour) are more complex. The SQL in example 4 illustrates these more complex feature selections.Example 4: intermediate view definition for adding featuresc. Highest-level viewHaving selected all necessary features in the prior view, it’s time to select the label. The label should be the strike range that would have been in-the-money for a given historical hashtag and ten-minute-window. The application’s “Contract Issuance” batch job generates strike ranges for every 10-minute window, and its “Expiration and Settlement” job determines which contract (range) struck in-the-money. When labeling historical examples for model training, it’s critical to apply this exact same application logic.Example 5: highest level view3. Train and get predictions from modelHaving created a view containing our features and label, we refer to the view in our BigQuery ML model creation statement:Example 6: model creationThen, at the time of contract issuance, we execute a query against the model to retrieve a prediction as to which contract will be in-the-money.Example 7: SELECTing predictions FROM the modeImprovementsThe exchange was built with a relatively short lead time, hence there were several architectural and tactical simplifications made in order to realistically ship on schedule. Future iterations of the exchange will look to implement several enhancements, such as:Introduce Cloud Pub/Sub into the architectureCloud Pub/Sub is an enabler for refined data pipeline architectures, and it stands to improve several areas within the exchange’s solution architecture. For example, it would reduce the latency of reported tweet counts by allowing the requisite components to be event-driven rather than batch-oriented.Replace VM `cron` jobs with Cloud SchedulerThe current architecture relies on Linux `cron`, running on a Compute Engine instance, for issuing and expiring options contracts, which contributes to the net administrative footprint of the solution. Launched in November of last year (after the version 1 architecture had been deployed), Cloud Scheduler will enable the team to provide comparable functionality with less infrastructural overhead.Reduce the size of the code base by leveraging Dataflow templatesOften, solutions contain non-trivial amounts of code responsible for simply moving data from one place to another, like persisting Pub/Sub messages to BigQuery. Cloud Dataflow templates allow development teams to shed these non-differentiating lines of code from their applications and simply configure and manage specific pipelines for many common use cases. Expand the stored attributes of ingested tweetsStoring the geographical tweet origins and the actual texts of ingested tweets could provide a richer basis from which future contracts may be defined. For example, sentiment analysis could be performed on the Tweet contents for particular hashtags, thus allowing binary contracts to be issued pertaining to the overall sentiment on a topic.Consider BigQuery user-defined functions (UDFs) to eliminate duplicate code among batch jobs and model executionCertain functionality, such as the ability to nimbly deal with time in 10-minute slices, is required by multiple pillars of the architecture, and resulted in the team deploying duplicate algorithms in both SQL and Javascript. With BigQuery UDFs, the team can author the algorithm once, in Javascript, and leverage the same code assets in both the Javascript batch processes as well as in the BigQuery ML models.A screenshot of the exchange dashboard during a trading sessionIf you’re interested in learning more about BigQuery ML, check out our documentation, or more broadly, have a look at our solutions for the financial services industry, or check out this interactive BigQuery ML walkthrough video. Or, if you’re able to attend Google Next ‘19 in San Francisco, you can even try out the exchange for yourself. Read more »
  • Build an AI-powered, customer service virtual agent with Chatbase
    These days, most people don’t tolerate more than one or two bad customer service experiences. For contact centers drowning in customer calls and live chats, an AI-powered customer service virtual agent can reduce that risk by complementing humans to provide personalized service 24/7, without queuing or waiting. But the status-quo approach to designing those solutions (i.e., intuition and brainstorming) is slow, based on guesswork, and just scratches the surface on functionality -- usually, causing more harm than good because the customer experience is poor.Built within Google’s internal incubator called Area 120, Chatbase is a conversational AI platform that replaces the risky status quo approach with a data-driven one based on Google’s world-class machine learning and search capabilities. The results include faster development (by up to 10x) of a more helpful and versatile virtual agent, and happier customers!Lessons learned along the journeyInitially, Chatbase provided a free-to-use analytics service for measuring and optimizing any AI-powered chatbot. (That product is now called Chatbase Virtual Agent Analytics.) After analyzing hundreds of thousands of bots and billions of messages in our first 18 months of existence, we had two revelations about how to help bot builders in a more impactful way: one, that customer service virtual agents would become the primary use case for the technology; and two, that using ML to glean insights from live-chat transcripts at scale would drastically shorten development time for those agents while creating a better consumer experience. With those lessons learned, Chatbase Virtual Agent Modeling (currently available via an EAP) was born.Virtual Agent Modeling explainedVirtual Agent Modeling (a component in the Cloud Contact Center AI solution) uses Google’s core strengths in ML and search to analyze thousands of transcripts, categorizing customer issues into “drivers” and then digging deeper to find specific intents (aka customer requests) per driver. For complex intents, Chatbase models simple yet rich flows developers can use to build a voice or chat virtual agent that handles up to 99% of interactions, responds helpfully to follow-up questions, and knows exactly when to do a hand-off to a live agent.In addition, the semantic search tool finds potentially thousands of training phrases per intent. When this analysis is complete, developers can export results to their virtual agent (via Dialogflow) -- cutting weeks, months, or even years from development time.Don’t settle for the slow status quoAccording to one Fortune 100 company upgrading its customer service virtual agent with Virtual Agent Modeling, “This new approach to virtual agent development moves at 200 mph, compared to 10-20 mph with current solutions.” Furthermore, it expects to nearly double the number of interactions its virtual agent can handle, from 53% of interactions to 92%.If you have at least 100,000 English-language live-chat transcripts available and plan to deploy or enhance either a voice or chat customer service virtual agent in 2019, Virtual Agent Modeling can help you get into that fast lane. Request your personal demo today! Read more »
  • Introducing Feast: an open source feature store for machine learning
    To operate machine learning systems at scale, teams need to have access to a wealth of feature data to both train their models, as well as to serve them in production. GO-JEK and Google Cloud are pleased to announce the release of Feast, an open source feature store that allows teams to manage, store, and discover features for use in machine learning projects.Developed jointly by GO-JEK and Google Cloud, Feast aims to solve a set of common challenges facing machine learning engineering teams by becoming an open, extensible, unified platform for feature storage. It gives teams the ability to define and publish features to this unified store, which in turn facilitates discovery and feature reuse across machine learning projects.“Feast is an essential component in building end-to-end machine learning systems at GO-JEK,” says Peter Richens, Senior Data Scientist at GO-JEK, “we are very excited to release it to the open source community. We worked closely with Google Cloud in the design and development of the product,  and this has yielded a robust system for the management of machine learning features, all the way from idea to production.”For production deployments, machine learning teams need a diverse set of systems working together. Kubeflow is a project dedicated to making these systems simple, portable and scalable and aims to deploy best-of-breed open-source systems for ML to diverse infrastructures. We are currently in the process of integrating Feast with Kubeflow to address the feature storage needs inherent in the machine learning lifecycle.The motivationFeature data are signals about a domain entity, e.g: for GO-JEK, we can have a driver entity and a feature of the daily count of trips completed. Other interesting features might be the distance between the driver and a destination, or the time of day. A combination of multiple features are used as inputs for a machine learning model.In large teams and environments, how features are maintained and served can diverge significantly across projects and this introduces infrastructure complexity, and can result in duplicated work.Typical challenges:Features not being reused: Features representing the same business concepts are being redeveloped many times, when existing work from other teams could have been reused.Feature definitions vary: Teams define features differently and there is no easy access to the documentation of a feature.Hard to serve up to date features: Combining streaming and batch derived features, and making them available for serving, requires expertise that not all teams have. Ingesting and serving features derived from streaming data often requires specialised infrastrastructure. As such, teams are deterred from making use of real time data.Inconsistency between training and serving: Training requires access to historical data, whereas models that serve predictions need the latest values. Inconsistencies arise when data is siloed into many independent systems requiring separate tooling.Our solutionFeast solves these challenges by providing a centralized platform on which to standardize the definition, storage and access of features for training and serving. It acts as a bridge between data engineering and machine learning.Feast handles the ingestion of feature data from both batch and streaming sources. It also manages both warehouse and serving databases for historical and the latest data. Using a Python SDK, users are able to generate training datasets from the feature warehouse. Once their model is deployed, they can use a client library to access feature data from the Feast Serving API.Feast provides the following:Discoverability and reuse of features: A centralized feature store allows organizations to build up a foundation of features that can be reused across projects. Teams are then able to utilize features developed by other teams, and as more features are added to the store it becomes easier and cheaper to build models.Access to features for training: Feast allows users to easily access historical feature data. This allows users to produce datasets of features for use in training models. ML practitioners can then focus more on modelling and less on feature engineering.Access to features in serving: Feature data is also available to models in production through a feature serving API. The serving API has been designed to provide low latency access to the latest feature values.Consistency between training and serving: Feast provides consistency by managing and unifying the ingestion of data from batch and streaming sources, using Apache Beam, into both the feature warehouse and feature serving stores. Users can query features in the warehouse and the serving API using the same set of feature identifiers.Standardization of features: Teams are able to capture documentation, metadata and metrics about features. This allows teams to communicate clearly about features, test features data, and determine if a feature is useful for a particular model.KubeflowThere is a growing ecosystem of tools that attempt to productionize machine learning. A key open source ML platform in this space is Kubeflow, which has focused on improving packaging, training, serving, orchestration, and evaluation of models. Companies that have built successful internal ML platforms have identified that standardizing feature definitions, storage, and access, was critical to that success.For this reason, Feast aims to be both deployable on Kubeflow and to integrate seamlessly with other Kubeflow components.  This includes a Python SDK for use with Kubeflow's Jupyter notebooks, as well as Kubeflow Pipelines.There is a Kubeflow GitHub issue here that allows for discussion of future Feast integration.How you can contributeFeast provides a consistent way to access features that can be passed into serving models, and to access features in batch for training. We hope that Feast can act as a bridge between your data engineering and machine learning teams, and we would love to hear your feedback via our GitHub project. For additional ways to contribute:Find the Feast project on GitHub repository hereJoin the Kubeflow community and find us on SlackLet the Feast begin! Read more »
  • A simple blueprint for building AI-powered customer service on GCP
    As a Google Cloud customer engineer based in Amsterdam, I work with a lot of banks and insurance companies in the Netherlands. All of them have this common requirement: to help customer service agents (many of whom are poorly trained interns due to the expense of hiring) handle large numbers of customer calls, especially at the end of the year when many consumers want to change or update their insurance plan.Most of these requests are predictable and easily resolved with the exchange of a small amount of information, which is a perfect use case for an AI-powered customer service agent. Virtual agents can provide non-queued service around the clock, and can easily be programmed to handle simple requests as well as do a hand-off to well-trained live agents for more complicated issues. Furthermore, a well-designed solution can help ensure that consumer requests, regardless of the channel in which they are received (phone, chat, IoT), are routed to the correct resource. As a result, in addition to the obvious customer satisfaction benefits, research says that virtual agents could help businesses in banking and healthcare alone trim costs collectively by $11 billion a year.In this post, I’ll provide an overview of a simple solution blueprint I designed that may inspire you to meet these objectives using GCP. Similar solutions that integrate with existing call center systems can be obtained through Cloud Contact Center AI partners, as well.Requirements and solutionAll businesses have the goal of making customer service effortless. With an AI-powered approach, a system can be designed that can accommodate consumers however they choose to reach out, whether by telephone, web chat, social media, mobile apps, or smart speaker.The particular approach described here covers three channels: web chat, the Google Assistant (on a Google Home), and telephone (through a telephony gateway). It also meets a few other requirements:Ability to optimize over time. If you know what questions consumers ask and how their sentiment changes during a conversation, the virtual agent (and thus customer satisfaction) can be improved over time.Protection of consumer privacy. Per GDPR, sensitive personal information can’t be revealed or stored.An easy deployment and management experience. It goes without saying that any company adopting cloud wants to avoid maintaining VMs, networks, and operating systems, as well as monolithic architecture. Thus the solution should take advantage of the ability to easily/automatically build, deploy, and publish updates.With Google Cloud, meeting these requirements is as easy as stitching a few components together. Let’s have a closer look.Technology stackThe diagram below provides a high-level overview; I’ll explain each piece in turn.DialogflowDialogflow Enterprise Edition, an emerging standard for building AI-powered conversational experiences across multiple channels, is the “brains” of this solution. My customers love it because it doesn’t require special natural language understanding skills; a team of content experts and UX designers are all you need to build a robust virtual agent for a simple use case. It also integrates well with other Google Cloud components, offers error reporting and debug information out of the box, and is available along with Google Cloud Support and SLA.As you can see in the architectural diagram, Dialogflow is integrated with the website channel through the Dialogflow SDK. Integration with the Google Assistant or the Phone Gateway simply requires flipping a switch during configuration.ChannelsWebsite: The website front-end and back-end are split into two separate Kubernetes containers. The website front-end is build with Angular, and the back-end container is based on Node.js with integration. Dialogflow has a Node.js client library, so text messages from the Angular app are passed to the Node.js server app via WebSocket in order to send it to the Dialogflow SDK.The Google Assistant: Actions on Google is a framework for creating software applications (a.k.a., “actions”) for the Google Assistant. Actions on Google is nicely integrated in Dialogflow: Just log in with your Google account and you can easily deploy your agent to the Google Assistant, enabling interactions on Android apps, via the Google Assistant app on iOS, or on Google Home.Phone: As mentioned in the introduction, if your plan is to integrate your virtual agent with an existing contact center call system, Google Cloud partners like Genesys, Twilio, and Avaya can help integrate Cloud Contact Center AI with their platforms. (For an overview, see this video from Genesys.) For startups and SMBs, the Dialogflow Phone Gateway feature (currently in beta) integrates a virtual agent with a Google Voice telephone number with just a few clicks, creating an “instant” customer service voice bot.AnalyticsWhether you’re building a full customer service AI system, a simple action for the Google Assistant, or anything in between, it’s important to know which questions/journeys are common, which responses are most satisfying, and if and when the virtual agent isn’t programmed to respond beyond a default “fallback” message. The diagram below shows the solution analytics architecture for addressing this need.Cloud Pub/Sub: Cloud Pub/Sub, a fully-managed, real-time publish/subscribe messaging service that sends and receives messages between independent applications, is the “glue” that holds the analytic components together. All transcripts (from voice calls or chats) are sent to Cloud Pub/Sub as a first step before analysis.Cloud Functions: Google Cloud Functions is a lightweight compute platform for creating single-purpose, standalone functions that respond to events without the need to manage a server or runtime environment. In this case, the event will be triggered by Cloud Pub/Sub: Every time a message arrives there through the subscriber endpoint, a cloud function will run the message through two Google Cloud services (see below) before storing it in Google BigQuery.Cloud Natural Language: This service reveals the structure of a text message; you can use it to extract information about people, places, or in this case, to detect the sentiment of a customer conversation. The API returns a sentiment level between 1 and -1.Cloud Data Loss Prevention: This service discovers and redacts any sensitive information such as addresses and telephone numbers remaining in transcripts before storage.BigQuery: BigQuery is Google Cloud’s serverless enterprise data warehouse, supporting super-fast SQL queries enabled by the massive processing power of Google's infrastructure. Using BigQuery you could combine your website data together with your chat logs. Imagine you can see that your customer browsed through one of your product webpages, and then interacted with a chatbot. Now you can answer them proactively with targeted deals.. Naturally, this analysis can be done through a third-party business intelligence tool like Tableau, with Google Data Studio, or through a homegrown web dashboard like the one shown below.Another use case would be to write a query that returns all chat messages that have a negative sentiment score:SELECT * from `chatanalytics.chatmessages` where SCORE < 0 ORDER BY SCORE ASCThis query also returns the session ID,  so you could then write a query to get the full chat transcript and explore why this customer became unhappy:SELECT * from `chatanalytics.chatmessages` where SESSION = '6OVkcIQg7QFvdc5EAAAs' ORDER BY POSTEDDeployment: Finally, you can use Cloud Build to easily build and deploy these containers to Google Kubernetes Engine with a single command in minutes. A simple YAML file in the project will specify how this all works. As a result, each component/container can be independently modified as needed.Chatbase (optional): It’s not included in this blueprint, but for a more robust approach, Chatbase Virtual Agent Analytics (which powers Dialogflow Analytics and is free to use) is also an option. In addition to tracking health KPIs, it provides deep insights into user messages and journeys through various reports combined with transcripts. Chatbase also lets you report across different channels/endpoints.ConclusionRecently, it took me just a couple of evenings to build a full demo of this solution.And going forward, I don’t need to worry about installing operating systems, patches, or software, nor about scaling for demand: whether I have 10 or hundreds of thousands of users talking to the bot, it will just work. If you’re exploring improving customer satisfaction with an AI-powered customer service virtual agent, hopefully this blueprint is a thought-provoking place to start! Read more »
  • Getting started with Cloud TPUs: An overview of online resources
    The foundation for machine learning is infrastructure that’s powerful enough to swiftly perform complex and intensive data computation. But for data scientists, ML practitioners, and researchers, building on-premises systems that enable this kind of work can be prohibitively costly and time-consuming. As a result, many turn to providers like Google Cloud because it’s simpler and more cost-effective to access that infrastructure in the cloud.The infrastructure that underpins Google Cloud was built to push the boundaries of what’s possible with machine learning—after all, we use it to apply ML to many of our own popular products, from Street View to Inbox Smart Reply to voice search. As a result, we’re always thinking of ways we can accelerate machine learning and make it more accessible and usable.One way we’ve done this is by designing our very own custom machine learning accelerators, ASIC chips we call tensor processing units, or TPUs. In 2017 we made TPUs available to our Google Cloud customers for their ML workloads, and since then, we’ve introduced preemptible pricing, made them available for services like Cloud Machine Learning Engine and Kubernetes Engine, and introduced our TPU Pods.While we’ve heard from many organizations that they’re excited by what’s possible with TPUs, we’ve also heard from some that are unsure of how to get started. Here’s an overview of everything you might want to know about TPUs—what they are, how you might apply them, and where to go to get started.I want a technical deep dive on TPUsTo give users a closer look inside our TPUs, we published an in-depth overview of our TPUs in 2017 based on our in-datacenter performance analysis whitepaper.At Next ‘18, “Programming ML Supercomputers: A Deep Dive on Cloud TPUs” covered the programming abstractions that allow you to run your models on CPUs, GPUs, and TPUs, from single devices up to entire Cloud TPU pods. “Accelerating machine learning with Google Cloud TPUs” from O’Reilly AI Conference in September, also takes you through a technical deep dive on TPUs, as well as how to program them.And finally, you can also learn more about what makes TPUs fine-tuned for deep learning and hyperparameter tuning using TPUs in Cloud ML Engine.I want to know how fast TPUs are, and what they might costIn December, we published the MLPerf 0.5 benchmark results which measure performance for training workloads across cloud providers and on-premise hardware platforms. The findings demonstrated that a full Cloud TPU v2 pod can deliver the same result in 7.9 minutes of training time that would take a single state-of-the-art GPU 26 hours.From a cost perspective, the results also revealed revealed a full Cloud TPU v2 Pod can cost 38% less, than training the same model to the same accuracy on an n1-standard-64 Google Cloud VM with eight V100 GPUs attached, and can complete the training task 27 times faster. We also shared more on why we think Google Cloud is the ideal platform to train machine learning models at any scale.I want to I want to understand the value of adopting TPUs for my businessThe Next ‘18 session Transforming Your Business with Cloud TPUs can help you identify business opportunities to pursue with Cloud TPUs across a variety of application domains, including image classification, object detection, machine translation, language modeling, speech recognition, and more.One example of a business already using TPUs is eBay. Visual search is an important way eBay customers quickly find what they’re looking for. But with more than a billion product listings, eBay has found training a large-scale visual search model is no easy task. As a result, they turned to Cloud TPUs. You can learn more by reading their blog or watching their presentation at Next '18.I want to quickly get started with TPUsThe Cloud TPU Quickstart sets you up to start using TPUs to accelerate specific TensorFlow machine learning workloads on Compute Engine, GKE, and Cloud ML Engine. You can also take advantage of our open source reference models and tools for Cloud TPUs. Or you can try out this Cloud TPU self-paced lab.I want to meet up with Google engineers and others in the AI community to learn moreIf you’re located in the San Francisco Bay Area, our AI Huddles provide a monthly, in-person place where you can find talks, workshops, tutorials, and hands-on labs for applying ML on GCP. At our November AI Huddle, for example, ML technical lead Lak Lakshmanan shared how to train state-of-the-art image and text classification models on TPUs. You can see a list of our upcoming huddles here.Want to keep learning? Visit our website, read our documentation, or give us feedback. Read more »
  • NVIDIA Tesla T4 GPUs now available in beta
    In November, we announced that Google Cloud Platform (GCP) was the first and only major cloud vendor to offer NVIDIA’s newest data center GPU, the Tesla T4, via a private alpha. Today, these T4 GPU instances are now available publicly in beta in Brazil, India, Netherlands, Singapore, Tokyo, and the United States. For Brazil, India, Japan, and Singapore, these are the first GPUs we have offered in those GCP regions.The T4 GPU is well suited for many machine learning, visualization and other GPU accelerated workloads. Each T4 comes with 16GB of GPU memory, offers the widest precision support (FP32, FP16, INT8 and INT4), includes NVIDIA Tensor Core and RTX real-time visualization technology and performs up to 260 TOPS1 of compute performance. Customers can create custom VM shapes that best meet their needs with up to four T4 GPUs, 96 vCPUs, 624GB of host memory and optionally up to 3TB of in-server local SSD. Our T4 GPU prices are as low as $0.29 per hour per GPU on Preemptible VM instances. On-demand instances start at $0.95 per hour per GPU, with up to a 30% discount with sustained use discounts. Committed use discounts are also available as well for the greatest savings for on-demand T4 GPU usage—talk with sales to learn more.Broadest GPU availabilityWe’ve distributed our T4 GPUs across the globe in eight regions, allowing you to provide low latency solutions to your customers no matter where they are. The T4 joins our NVIDIA K80, P4, P100 and V100 GPU offerings, providing customers with a wide selection of hardware-accelerated compute options. T4 GPUs are now available in the following regions: us-central1, us-west1, us-east1, asia-northeast1, asia-south1, asia-southeast1, europe-west4,  and southamerica-east1.Machine learning inferenceThe T4 is the best GPU in our product portfolio for running inference workloads. Its high performance characteristics for FP16, INT8 and INT4 allow you to run high scale inference with flexible accuracy/performance tradeoffs that are not available on any other GPU. The T4’s 16GB of memory supports large ML models or running inference on multiple smaller models simultaneously. ML inference performance on Google Compute Engine’s T4s has been measured at up to 4267 images/sec2 with latency as low as 1.1ms3. Running production workloads on T4 GPUs on Compute Engine is a great solution thanks to the T4’s price, performance, global availability across eight regions and high-speed Google network. To help you get started with ML inference on the T4 GPU, we also have a technical tutorial demonstrating how to deploy a multi-zone, auto-scaling ML inference service on top of Compute Engine VMs and T4 GPUs.Machine learning trainingThe V100 GPU has become the primary GPU for ML training workloads in the cloud thanks to its high performance, Tensor Core technology and 16GB of GPU memory to support larger ML models. The T4 supports all of this at a lower price point,  making it a great choice for scale-out distributed training or when a V100 GPU’s power is overkill. Our customers tell us they like the near-linear scaling of many training workloads on our T4 GPUs as they speed up their training results with large numbers of T4 GPUs.ML cost savings options only on Compute EngineOur T4 GPUs complement our V100 GPU offering nicely. You can scale up with large VMs up to eight V100 GPUs, scale down with lower cost T4 GPUs or scale out with either T4 or V100 GPUs based on your workload characteristics. With Google Cloud as the only major cloud provider to offer T4 GPUs, our broad product portfolio lets you save money or do more with the same resources.* Prices listed are current Compute Engine on-demand pricing for certain regions.Prices may vary by region and lower prices are available through SUDs and Preemptible GPUsStrong visualization with RTXThe NVIDIA T4 with its Turing architecture is the first data center GPU to include dedicated ray-tracing processors. Called RT Cores, they accelerate the computation of how light travels in 3D environments. Turing accelerates real-time ray tracing over the previous-generation NVIDIA Pascal architecture and can render final frames for film effects faster than CPUs, providing hardware-accelerated ray tracing capabilities via NVIDIA’s OptiX ray-tracing API. In addition, we are glad to also offer virtual workstations running on T4 GPUs that give creative and technical professionals the power of the next generation of computer graphics with the flexibility to work from anywhere and on any device.Getting startedWe make it easy to get started with T4 GPUs for ML, compute and visualization. Check out our GPU product page to learn more about the T4 and our other GPU offerings. For those looking to get up and running quickly with GPUs and Compute Engine, our Deep Learning VMimage comes with NVIDIA drivers and various ML libraries pre-installed. Not a Google Cloud customer? Sign up today and take advantage of our $300 free tier.Want a demo? Watch our on-demand webinar to discover the eight reasons why to run your ML training and inference with NVIDIA T4 GPUs on GCP.1. 260 TOPs INT4 performance, 130 TOPs INT8, 65 TFLOPS FP16, 8.1 TFLOPS FP322. INT8 precision, resnet50, batch size 1283. INT8 precision, resnet50, batch size 1 Read more »
  • Running TensorFlow inference workloads at scale with TensorRT 5 and NVIDIA T4 GPUs
    Today, we announced that Google Compute Engine now offers machine types with NVIDIA T4 GPUs, to accelerate a variety of cloud workloads, including high-performance computing, deep learning training and inference, broader machine learning (ML) workloads, data analytics, and graphics rendering.In addition to its GPU hardware, NVIDIA also offers tools to help developers make the best use of their infrastructure. NVIDIA TensorRT is a software inference platform for developing high-performance deep learning inference—the stage in the machine learning process where a trained model is used, typically in a runtime, live environment, to recognize, process, and classify results. The library includes a deep learning inference data type (quantization) optimizer, model conversion process, and runtime that delivers low latency and high throughput. TensorRT-based applications perform up to 40 times faster1 than CPU-only platforms during inference. With TensorRT, you can optimize neural network models trained in most major frameworks, calibrate for lower precision with high accuracy, and finally, deploy to a variety of environments. These might include hyperscale data centers, embedded systems, or automotive product platforms.In this blog post, we’ll show you how to run deep learning inference on large-scale workloads with NVIDIA TensorRT 5 running on Compute Engine VMs configured with our Cloud Deep Learning VM image and NVIDIA T4 GPUs.OverviewThis tutorial shows you how to set up a multi-zone cluster for running an inference workload on an autoscaling group that scales to meet changing GPU utilization demands, and covers the following steps:Preparing a model using a pre-trained graph (ResNet)Benchmarking the inference speed for a model with different optimization modesConverting a custom model to TensorRT formatSetting up a multi-zone cluster that is:Built on Deep Learning VMs preinstalled with TensorFlow, TensorFlow serving, and TensorRT 5.Configured to auto-scale based on GPU utilization.Configured for load-balancing.Firewall enabled.  Running an inference workload in the multi-zone cluster.Here’s a high-level architectural perspective for this setup:Preparing and optimizing the model with TensorRTIn this section, we will create a VM instance to run the model, and then download a model from the TensorFlow official models catalog.Create a new Deep Learning Virtual Machine instanceCreate the VM instance:If command is successful you should see a message that looks like this:Notes:You can create this instance in any available zone that supports T4 GPUs.A single GPU is enough to compare the different TensorRT optimization modes.Download a ResNet model pre-trained graphThis tutorial uses the ResNet model, which trained on the ImageNet dataset that is in TensorFlow. To download the ResNet model to your VM instance, run the following command:Verify model was downloaded correctly:Save the location of your ResNet model in the $WORKDIR variable:Benchmarking the modelLeveraging fast linear algebra libraries and hand tuned kernels, TensorRT can speed up inference workloads, but the most significant speed-up comes from the quantization process. Model quantization is the process by which you reduce the precision of weights for a model. For example, if the initial weight of a model is FP32, you have the option to reduce the precision to FP16 and INT8 with the goal of improving runtime performance. It’s important to pick the right balance between speed (precision of weights) and accuracy of a model. Luckily, TensorFlow includes functionality that does exactly this, measuring accuracy vs. speed, or other metrics such as throughput, latency, node conversion rates, and total training time. TensorRT supports two modes: TensorFlow+TensorRT and TensorRT native, in this example we use the first option.Note: This test is limited to image recognition models at the moment, however it should not be too hard to implement a custom test based on this code.Set up the ResNet modelTo set up the model, run the following command:This test requires a frozen graph from the ResNet model (the same one that we downloaded before), as well as arguments for the different quantization modes that we want to test.The following command prepares the test for the execution:Run the testThis command will take some time to finish.Notes:$WORKDIR is the directory in which you downloaded the ResNet model.The --native arguments are the different available quantization modes you can test.Review the resultsWhen the test completes, you will see a comparison of the inference results for each optimization mode.To see the full results, run the following command:V100T4P4From the above results, you can see that FP32 and FP16 performance numbers are identical under predictions. This means that if you are content working with TensorRT, you can definitely start using FP16 right away. INT8, on the other hand, shows slightly worse accuracy and requires understanding the accuracy-versus-performance tradeoffs for your models.In addition, you can observe that when you run the model with TensorRT 5:Using FP32 optimization improves throughput by 40% (440 vs 314). At the same time it decreases latency by ~30%, making it 0.28 ms instead of 0.40 ms.Using FP16 optimization rather than native TF graph increases the speed by 214%! (from 314 to 988 fps). At the same time latency decreased by 0.12 ms (almost 3x decrease!).Using INT8, the last result displayed above, we observed a speedup of 385% (from 314 to 1524) with the latency decreasing to 0.08 ms.Notes:The above results do not include latency for image pre-processing nor HTTP requests latency. In production systems the inference’ speed may not be a bottleneck at all, and you will need to account for all the factors mentioned in order to measure your end to end inference’ speed.Now, let’s pick a model, in this case, INT8.Converting a custom model to TensorRTDownload and extract ResNet modelTo convert a custom model to a TensorRT graph you will need a saved model. To download a saved INT8 ResNet model, run the following command:SavedModels are generated to accept either tensor or JPG inputs, and with channels_first (NCHW) and channels_last (NHWC) convolutions. NCHW is generally better for GPUs, while NHWC is generally better for CPUs, in this case we are downloading a model that can handle JPG inputs.Convert the model to a TensorRT graph with TFToolsNow we can convert this model to its corresponding TensorRT graph with a simple tool:You now have an INT8 model in your $WORKDIR/resnet_v2_int8_NCHW/00001 directory.To ensure that everything is set up properly, try running an inference test.Upload the model to Cloud StorageYou’ll need to run this step so that the model can be served from the multi-zone cluster that we will set up in the next section. To upload the model, complete the following steps:1. Archive the model.2. Upload the archive.If needed, you can obtain an INT8 precision variant of the frozen graph from Cloud Storage at this URL:Setting up a multi-zone clusterCreate the clusterNow that we have a model in Cloud Storage, let’s create a cluster.Create an instance templateAn instance template is a useful way to create new instances. Here’s how:Notes:This instance template includes a startup script that is specified by the metadata parameter.The startup script runs during instance creation on every instance that uses this template, and performs the following steps:Installs NVIDIA drivers, NVIDIA drivers are installed on each new instance. Without NVIDIA drivers, inference will not work.Installs a monitoring agent that monitors GPU usage on the instanceDownloads the modelStarts the inference serviceThe startup script runs, which contains the inference logic. For this example, I have created a very small Python file based on the TFServe package.To view the startup script, see a managed instance groupYou’ll need to set up a managed instance group, to allow you to run multiple instances in specific zones. The instances are created based on the instance template generated in the previous step.Notes:INSTANCE_TEMPLATE_NAME is the name of the instance that you created in the previous step.You can create this instance in any available zone that supports T4 GPUs. Ensure that you have available GPU quotas in the zone.Creating the instance takes some time. You can watch the progress with the following command:Once the managed instance group is created, you should see output that resembles the following:Confirm metrics in Stackdriver1. Access Stackdriver’s Metrics Explorer here2. Search for gpu_utilization. StackDriver > Resources > Metrics Explorer3. If data is coming in, you should see something like this:Enable auto-scalingNow, you’ll need to enable auto-scaling for your managed instance group.Notes:The is the full path to our metric.We are using level 85, this means that whenever GPU utilization reaches 85, the platform will create a new instance in our group.Test auto-scalingTo test auto-scaling, perform the following steps:1. SSH to the instances. See Connecting to Instances for more details.2. Use the gpu-burn tool to load your GPU to 100% utilization for 600 seconds:Notes:During the make process, you may receive some warnings, ignore them.You can monitor the gpu usage information, with a refresh interval of 5 seconds:3. You can observe the autoscaling in Stackdriver, one instance at a time.4. Go to the Instance Groups page in the Google Cloud Console.5. Click on the deeplearning-instance-group managed instance group.6. Click on the Monitoring tab.At this point your auto-scaling logic should be trying to spin as many instances as possible to reduce the load. And that is exactly what is happening:At this point you can safely stop any loaded instances (due to the burn-in tool) and watch the cluster scale down.Set up a load balancerLet's revisit what we have so far:A trained model, optimized with TensorRT 5 (using INT8 quantization)A managed instance group. These instances have auto-scaling enable based on the GPU utilizationNow you can create a load balancer in front of the instances.Create health checksHealth checks are used to determine if a particular host on our backend can serve the traffic.Create inferences forwarderConfigure named-ports of the instance group so that LB can forward inference requests, sent via port 80, to the inference service that is served via port 8888.Create a backend serviceCreate a backend service that has an instance group and health check.First, create the health check:Then, add the instance group to the new backend service:Set up the forwarding URLThe load balancer needs to know which URL can be forwarded to the backend services.Create the load balancerAdd an external IP address to the load balancer:Find the allocated IP address:Set up the forwarding rule that tells GCP to forward all requests from the public IP to the load balancer:After creating the global forwarding rules, it can take several minutes for your configuration to propagate.Enable the firewallYou need to enable a firewall on your project, or else it will be impossible to connect to your VM instances from the external internet. To enable a firewall for your instances, run the following command:Running inferenceYou can use the following Python script to convert images to a format that can be uploaded to the server.Finally, run the inference request:That’s it!Toward TensorFlow inference blissRunning ML inference workloads with TensorFlow has come a long way. Together, the combination of NVIDIA T4 GPUs and its TensorRT framework make running inference workloads a relatively trivial task—and with T4 GPUs available on Google Cloud, you can spin them up and down on demand. If you have feedback on this post, please reach out to us here.Acknowledgements: Viacheslav Kovalevskyi, Software Engineer, Gonzalo Gasca Meza, Developer Programs Engineer, Yaboo Oyabu, Machine Learning Specialist and Karthik Ramasamy, Software Engineer contributed to this post.1. Inference benchmarks show ResNet training times to be 27x faster, and GNMT times to be 36x faster Read more »
  • Coastal classifiers: using AutoML Vision to assess and track environmental change
    Tracking changes in the coastline and its appearance is an effective means for many scientists to monitor both conservation efforts and the effects of climate change. That’s why the Harte Research Institute at TAMUCC (Texas A&M University - Corpus Christi) decided to use Google Cloud’s AutoML Vision classifiers to identify attributes in large data sets of coastline imagery, in this case, of the coastline along the Gulf of Mexico. This post will describe how AutoML’s UI helped TAMUCC’s researchers improve their model’s accuracy, by making it much easier to build custom image classification models on their own image data. Of course not every organization wants to analyze and classify aerial photography, but the techniques discussed in this post have much wider applications, for example industrial quality control and even endangered species detection. Perhaps your business has a use case that can benefit from AutoML Vision’s custom image classification capabilities.The research problem: classification of shoreline imageryThe researchers at the Harte Research Institute set out to identify the types of shorelines within aerial imagery of the coast, in order to accurately predict the Environmental Sensitivity Index (ESI) of shorelines displayed in the images, which indicate how sensitive a section of shoreline would be to an oil spill.   Anthony Reisinger and his colleagues at the Harte developed an Environmental Sensitivity Index map of shorelines that may be impacted by oil spills for the State (government) of Texas. During this process, the team looked at oblique aerial photos and orthophotos similar to what one might find on Google Maps, and manually traced out shorelines for the entire length (8950 miles) of the Texas coast (see below). After the team traced the shoreline, they then coded it with ESI values that indicate how sensitive the shoreline is to oil. These values were previously standardized by experts who had spent many years in the field and scrutinizing coastal images.Texas coast with cyan overlay of ESI shorelineAfter an oil spill, the State of Texas uses these ESI shoreline classifications to send out field crews to highly sensitive environments near the oil spill. The State then isolates sensitive habitats with floating booms (barriers that float on water and extend below the surface) to minimize the oil's impact on the environment and the animals that live there.As you might imagine, the process of learning how to identify the different types of environment classifications and how sensitive these shorelines are to oil spills takes years of first-hand experience, especially when imagery is only available at different scales and resolutions. Some of the team’s research over the years has utilized machine learning, so the researchers decided to see if their expert knowledge could be transferred over to a machine and automate the identification of the different types of ESI shorelines within the images and among the different types of imagery used.  Coastal environments can rapidly change due to natural processes as well as coastal development, thus, the state needs to update its ESI shoreline assessments periodically. At the moment, the team plans to update the ESI shoreline data set for the entire Gulf Coast that lies within the State of Texas. During this process, new oblique imagery will be acquired to help identify the shorelines sensitivity to oil spills. With AutoML Vision, the team takes newly-acquired oblique imagery and predicts the ESI values in the shoreline photos, thereby classifying (or coding) the new shoreline file we create.Imagery typesThe team experimented with two different types of aerial shoreline images: oblique and orthorectified. For orthorectified aerial photos, a grid was overlaid on the imagery and ESI shorelines, and both were extracted for each grid cell and joined together. For the oblique shoreline photos, the team experimented with applying both single labels and multiple labels (also known as multi-class detection). Details of this latter approach are mentioned later in this post.Rectified imagery overlaid with ESI shorelines and grid used to extract both imagery and shorelines.But to begin, let’s take a look at the results on the oblique image set. Early experiments comparing precision and recall metrics of AutoML models using orthorectified aerial photos of different pixel resolutions and acquisition dates were compared to AutoML models of oblique photos. Interestingly, the team found that prediction accuracy using oblique imagery models was higher than that for orthorectified imagery models. The oblique imagery models' higher performance is likely due to larger geographic coverage of the oblique imagery, and the inclusion of vertical information in these images.Cloud Vision’s limitations for our use caseA little testing confirmed that the out-of-the-box Cloud Vision API won’t help with this task: Cloud Vision can identify many image categories, but, unsurprisingly, the results proved too general for the team’s purposes.Cloud Vision’s labels are too general for coastline classification.The team then decided that the shoreline images dataset is perfect for use with AutoML Vision, which let the team build their own domain-specific classifier.Cloud AutoML Vision provides added flexibilityCloud AutoML allows developers with limited machine learning expertise to train high-quality models specific to their data and business needs, by leveraging Google’s state-of-the-art transfer learning and Neural Architecture Search technology. Google’s suite of AutoML products currently includes Natural Language and Translate as well as Vision, all currently in beta.By using AutoML Vision, the team was able to train custom image classification models with only a labeled dataset of images. AutoML does all the rest for you: it trains advanced models using your data, lets you inspect your data and analyze the results via an intuitive UI, and provides an API for scalable serving.The first AutoML experiment: single-labeled imagesThe team first experimented with a single-label version of the oblique image set, in which the label referred to a single primary shoreline type included in the image. The quality of this model was passable, but not as accurate as the team had hoped. To generate the image labels, the direction of the camera and aircraft position were used to project a point to the closest shoreline, and each image was assigned a label based on both the camera’s heading and the proximity of the shoreline to the plane’s position.Map showing single-label method used to join the aircraft’s position with the nearest ESI shoreline from the image on the left. Image was taken from the aircraft’s position in the map (Note: the projected point was assigned the value of the closest shoreline point to the plane/camera’s location; however, this photo contains multiple shoreline types).Precision and recall metrics across all labels, using the single-label dataset.The AutoML UI allows easy visual inspection of your model’s training process and evaluation results, under the Evaluate tab. You can look at the metrics for all images, or focus on the results for a given label, including its true positives, false positives, and false negatives. Thumbnail renderings of the images allow you to quickly scan each category.From inspection of the false positives, the team was able to determine that this first model often predicted coastline types that were actually present in the image, but did not match the single label. Below, you can see one example, in which the given coastline label was salt_brackish_water_marshes, but the model predicted gravel_shell_beaches with higher probability. In fact, the image does show gravel shell beaches as well as marshes.AutoML Vision correctly predicts that this image contains gravel shell beaches, even though it wasn’t labeled as such.After examining these evaluation results, the team concluded that this data would be a better fit for multi-label classification, in which a given image could be labeled as containing more than one type of shoreline. (Similarly, you might want to apply multiple classes to your training data as well, depending on your use case.)The second AutoML experiment: multi-labeled imagesAutoML supports multi-labeled datasets, and enables you to train and use such models. With this capability in mind, the team soon discovered it was possible to generate such a dataset from the original source images, and then ran a second set of experiments using the same images, but tagged with multiple labels per image, where possible. This dataset resulted in significantly more accurate models than those built using the single-label dataset.Map illustrating the multi-label method of ESI shoreline label extraction using modeled field-of-view (FOV) of the camera and the image taken from the aircraft’s position on the map (Note: this method allows for the majority of the shorelines in the FOV to be assigned to this image).Precision and recall metrics across all labels, for a model trained on the multi-label dataset.The following image is representative of how the multi-labeling helped: its labels include both gravel_shell_beaches and salt_brackish_water_marshes, and it correctly predicts both.This image’s multiple classifications were correctly predicted.Viewing evaluation results and metricsAutoML’s user interface (UI) makes it easy to view evaluation results and metrics. In addition to overall metrics, you can view how well the model performed with each label, including display of representative true positives, false positives, and false negatives. A slider lets you adjust the score threshold for classification—for all labels or for just a single label—and then observe how the precision-recall tradeoff curve changes in response.Often, classification accuracy is higher for some labels than others, especially if your dataset includes some bias. This information can be useful in determining whether you might increase model accuracy by sourcing additional images for some of your labels, then further training (or retraining) them.Viewing evaluation results for the “gravel_shell_beaches” label.Comparing models built using the same datasetAutoML Vision allows you to indicate how much (initial1) compute time to devote to creating a model. As part of the team’s experimentation, it also compared two models, the first created using one hour of compute time, and the other using 24 hours. As expected, the latter model was significantly more accurate than the former. (This was the case for the single-label dataset as well.)Viewing evaluation results for the “gravel_shell_beaches” label.Using your models for predictionAutoML Vision makes it easy for you to use your trained models for prediction. You can use the Predict tab in the UI to see visually how a model is doing on a few images. You can use the ‘export data’ feature in the top navigation bar to see which of your images were in which data set (training, validation, or test) to avoid using training images.Predicting the classes of shoreline shown in a new imageThen, you can access your model via its REST API for scalable serving, either programmatically or from the command line. The Predict tab in the AutoML UI includes examples of how to do this.What’s nextWe hope this helps demonstrate how Cloud AutoML Vision can be used to accurately classify different types of shorelines in aerial images. We plan to create an updated version of the ESI shoreline dataset in the future and use the AutoML model to predict shoreline types on newly acquired oblique photography and orthorectified imagery. Use of AutoML will allow non-experts the ability to assign ESI values to these shorelines we create. Try it yourselfThe datasets we used in these experiments are courtesy of the Harte Research Institute at the Texas A&M University - Corpus Christi. You can use these datasets yourself. See this README for more information, and see this documentation page for permission details. Of course you can use the same techniques to classify other types of geographical or geological features, or even entirely unrelated image categories. AutoML Vision lets you extend and retrain the models that back the Cloud Vision API with additional classes, on data from your organization’s use case.AcknowledgementsThanks to Philippe Tissot, Associate Director, Conrad Blucher Institute for Surveying and Science, Texas A&M University - Corpus Christi; James Gibeaut, Endowed Chair for Coastal and Marine Geospatial Sciences, Harte Research Institute, Texas A&M University - Corpus Christi; and Valliappa Lakshmanan, Tech Lead, Google Big Data and Machine Learning Professional Services, for their contributions to this work.1. For options other than the default ‘1 hour’ of compute time, model training can be resumed later if desired. If you like, you can add additional images to the dataset before resumption. Read more »
  • Using data and ML to better track wildfire and assess its threat levels
    As California’s recent wildfires have shown, it’s often hard to predict where fire will travel. While firefighters rely heavily on third-party weather data sources like NOAA, they often benefit from complementing weather data with other sources of information. (In fact, there’s a good chance there’s no nearby weather station to actively monitor weather properties in and around a wildfire.) How, then, is it possible to leverage modern technology to help firefighters plan for and contain blazes?Last June, we chatted with Aditya Shah and Sanjana Shah, two students at Monta Vista High School in Cupertino, California, who’ve been using machine learning in an effort to better predict the future path of a wildfire. These high school seniors had set about building a fire estimator, based on a model trained inTensorFlow, that measures the amount of dead fuel on the forest floor—a major wildfire risk. This month we checked back in with them to learn more on how they did it.Why pick this challenge?Aditya spends a fair bit of time outdoors in the Rancho San Antonio Open Space Preserve near where he lives, and wanted to protect it and other areas of natural beauty so close to home. Meanwhile, after being evacuated from Lawrence Berkeley National Lab in the summer of 2017 due a nearby wildfire, Sanjana wanted to find a technical solution that reduced the risk of fire even before it occurs. Wildfires not only destroy natural habitat but also displace people, impact jobs, and cause extensive damage to homes and other property. Just as prevention is better than a cure, preventing a potential wildfire from occurring is more effective than fighting it.With a common goal, the two joined forces to explore available technologies that might prove useful. They began by taking photos of the underbrush around the Rancho San Antonio Open Space Preserve, cataloguing a broad spectrum of brush samples—from dry and easily ignited, to green or wet brush, which would not ignite as easily. In all, they captured 200 photos across three categories of underbrush: “gr1” (humid), “gr2” (dry shrubs and leaves), and “gr3” (no fuel, plain dirt/soil, or burnt fuel).Aditya and Sanjana then trained a successful model with 150 sample (training) images (roughly 50 in each of the three classes) plus a 50 image test (evaluation) set. For training, the pair turned to Keras, their preferred Python-based, easy-to-use deep learning library. Training the model in Keras has two benefits—it permits you to export to a TensorFlow estimator, which you can run on a variety of platforms and devices, and it allows for easy and fast prototyping since it runs seamlessly on either CPU or GPU.Preparing the dataBefore training the model, Aditya and Sanjana ran a preprocessing step on the data: resizing and flattening the images. They used the image_to_function_vector, which accepts raw pixel intensities from an input bitmap image and resizes that image to a fixed size, to ensure each image in the input dataset has the same ‘feature vector’ size. As many of the images are of different sizes, the pair resized all their captured images to 32x32 pixels. Since Keras models take as their input a 1-dimensional feature vector, he needed to flatten the 32x32x3 image into a 3072-dimensional feature vector. Further, he defined the ImagePaths to initiate the list of data and label, then looped over the ImagePaths individually to load them to the folder storage using cv2.imread function. Next, the pair extracted the class labels (as gr1, gr2, and gr3) from each image’s name. He then converted the images to feature vectors using image_to_feature_vector function and updated the data and label lists to match.Aditya and Sanjana next discovered that the simplest way to build the model was to linearly stacks layers, to form a sequential model, which simplified organization of the hidden layers. They were able to use the img2vec function, built into TensorFlow, as well as a support-vector-machine (SVM) layer.Next, the pair trained the model using a stochastic gradient descent (SGD) optimizer with learning rate = 0.05. SGD is an iterative method for finding an optimal solution by using stochastic gradient descent. There are a number of gradient descent methods typically used, including rmsprop, adagrad, adadelta, and adam. Aditya and Sanjana tried rmsprop, which yielded very low accuracy (~47%). Some methods like adadelta and adagrad yielded slightly higher accuracy but took more time to run. So they decided to use SGD as it offers better optimization with good accuracy and fast running time. In terms of hyperparameters, the pair tried different numbers of training epochs (50, 100, 200) and batch_size values (10, 35, 50), and he achieved the highest accuracy (94%) with epoch = 200 and batch_size = 35.In the testing phase, Aditya and Sanjana were able to achieve the 94.11% accuracy utilizing only the raw pixel intensities of the fuel images.The biggest challenge in this whole process was the data pre-processing step, as it involved accurately labeling the images. This very tedious task took Aditya and Sanjana more than four weeks while they created their training dataset.Modeling fire based on sensor dataAlthough Aditya and Sanjana now had a way to classify whether different types of underbrush were ripe for fire, they wondered how they might assess the current status of an entire area of land like the Rancho San Antonio Open Space Preserve.To do it, the pair settled on a high-definition image sensor, which connects over long-range, low-power LTE, to capture and relay images to Aditya’s computer, where they can run the model and classify the new inbound images. The device also collects a number of other metrics, including wind speed, wind direction, humidity, and temperature, and can classify an area of roughly 100 square meters (during daylight hours) to determine whether the ground cover will likely ignite or not. Aditya and Sanjana are currently collecting sensor data and testing the accuracy of their model at five different sites.Sensor and ML classifier architecture diagramBy combining real-time humidity and temperature data with NOAA-based wind speed and direction estimates, Aditya and Sanjana hope to determine in which direction a fire might travel. For the moment, the image-classification and the sensor-based prediction systems operate independently, but they plan to combine them in the future.What’s nextAlthough currently the pair are simply running TensorFlow models on Aditya’s gaming notebook PC (a tool of choice for data scientists on the go), Aditya and Sanjana plan to try out Cloud ML Engine in the future, to enable more flexible scaling than was possible on a single laptop. Image gathering from remote forest areas is another challenge they want to tackle, and they’re experimenting with all-terrain ground and aerial drones to collect data for this purpose.The pair also plan to continue their efforts working with Cal Fire to deploy their device. Currently Cal Fire determines moisture levels by weighing known fuels, such as wood sticks, a process that requires human labor. Aditya and Sanjana’s device offers the potential to reduce the need for that labor.Although fire is an often devastating force of nature, it is inspiring that a team of high schoolers hope to provide an effective new tool to assist agencies in predicting and tracking both wildfires and the weather factors that enable them to spread. Read more »
  • Where poetry meets tech: Building a visual storytelling experience with the Google Cloud Speech-to-Text API
    This post is a Q&A with Developer Advocate Sara Robinson and Maxwell Neely-Cohen, a Brooklyn-based writer who used the Cloud Speech-to-Text API to take words spoken at a poetry reading and display them on a screen to enhance the audience’s experience.Could you tell me about the oral storytelling problem you’re trying to solve?When you become a writer, you end up spending a lot of time at readings, where authors read their work aloud live to an audience. And I always wondered if it might be possible to create dynamic reactive visuals for a live reading the same way you might for a live musical performance. I didn’t have a specific design or aesthetic or even goal in mind, nor did I think the experience would necessarily be the greatest thing ever, I just wanted to see if it was possible. How it might work or what it might look like? That was my question. While the result was systemically very simple, sending speech-to-text results through a dictionary that had been sorted by what color people thought words were, it ended up being the perfect test case for this sort of idea and a ton of fun to play with.What is your background?I’m a novelist in the very old school dead tree literary fiction sense, but a lot of what I write about involves the social and cultural impact of technology. My first novel had a lot of internet and video game culture embedded in it, so I’ve always been following and thinking about software development even without that ever being my life or career. I did a whole bunch of professional music production and performance when I was a teenager and in college. This experience gave me at least a little bit of a technical relationship to using hardware and software creatively and it was the main reason I felt confident enough to undertake this project. Lately I’ve been doing these little projects to try to get the literary world and the tech world in greater conversation and collaboration. I think those two cultures have a lot to offer each other.How did you come across the Google Cloud Speech-to-Text API, and what makes it a good fit for adding visuals to poetry?We ended up searching for every possible speech-to-text API or software that exists, and tried to find the one that reacted fastest with the least possible amount of lag. We had been messing around with a few others, and decided to give Cloud Speech-to-Text a try, and it just worked beautifully. Because the API can so quickly return an interim result, a guess, in addition to a final updated guess, it was really ideal for this project. We had been kind of floundering for a day, and then it was like BAM as soon as the API got involved.What’s the format of these poetry events? Could you tell me more about CultureHub?The first weeklong residency, last June, was four days of development with an absolute genius NYU ITP student named Oren Shoham, and then three days of having writers come in and test it. I just emailed a whole bunch of friends basically, who luckily for me includes a lot of award-winning authors, and they were kind enough to show up and launch themselves into it.  We really had no idea what would work and what wouldn’t, so it was just a very experimental process. The second week, this November, we got the API running into Unity, and then had a group of young game developers prototype different visual designs for the system. They spent four days cranking out little ideas, and then we had public event, a reading with poets Meghann Plunkett, Rhiannon McGavin, Angel Nafis, and playwright Jeremy O. Harris, just to see what it would be like to have in the context of an event. Both times I tried to create collaborative environments, so it wasn’t just me trying to do it all myself. With experimental creative forms, I think having as many viewpoints in the room as possible is important. CultureHub is a collaboration between the famed La MaMa Experimental Theatre Club and Seoul Institute of the Arts. It’s a global art community that supports and curates all sorts of work using emerging technologies. They are particularly notable for projects that have used telepresence in all sorts of creative ways. It’s a really great place to try out an idea like this, something there previously wasn’t a great analog for.How did you solve this with Cloud Speech-to-Text? Any code snippets you can share?For the initial version, we used a Python script to interact with the API, the biggest change being adapting and optimizing it to run pseudo-continuously, then feeding the results into the NRC Word-Emotion Association Lexicon, a database which had been assembled by computer scientists Saif Mohammad and Peter Turney. We then fed both the color results and the text itself into a Max/MSP patch which generated the visual results. The second version used Node instead of the Python script, and Unity instead of Max/MSP. You can find it on GitHub.Do you have advice for other new developers looking to get started with Cloud Speech-to-Text or ML in general?I would say even if you have no experience coding, if you have an idea, just go after it. Building collaborative environments where non-technical creatives can collaborate with developers is innovative and fun in itself. I would also say there can be tremendous value in ideas that have no commercial angle or prospect. No part of what I wanted to do was a potential business idea or anything like that, it’s just a pure art project done because why not.Have questions for Max? Find him on Twitter @nhyphenc, and check out the GitHub repo for this project here. Read more »
  • Announcing the beta release of SparkR job types in Cloud Dataproc
    The R programming language is often used for building data analysis tools and statistical apps, and cloud computing has opened up new opportunities for those of you developing with R. Google Cloud Platform (GCP) lets you store a massive amount of data cost-effectively, then take advantage of managed compute resources only when you’re ready to scale your analysis.We’re pleased to announce the beta release of SparkR jobs on Cloud Dataproc, which is the latest chapter in building R support on GCP. SparkR is a package that provides a lightweight front end to use Apache Spark from R. This integration lets R developers use dplyr-like operations on datasets of nearly any size stored in Cloud Storage. SparkR also supports distributed machine learning using MLlib. You can use this integration to process against large Cloud Storage datasets or perform computationally intensive work.Cloud Dataproc is GCP’s fully managed cloud service for running Apache Spark and Apache Hadoop clusters in a simple and cost-efficient way. The Cloud Dataproc Jobs API makes it easy to submit SparkR jobs to a cluster without having to open firewalls to access web-based IDEs or SSH directly onto the master node. With the Jobs API, you can automate the repeatable R statistics you want to run on your datasets.Using GCP for R lets you avoid the infrastructure barriers that used to impose limits on understanding your data, such as choosing which datasets to sample because of compute or data size limits. With GCP, you can build large-scale models to analyze datasets of sizes that previously would have required huge upfront investments in high-performance computing infrastructures.What happens when you move R to the cloud?When you move R to the cloud, you’ll see the familiar RStudio interface that has served you well in prior R endeavors. While the SparkR Jobs API offers an easy way to execute SparkR code and automate tasks, most R developers will still perform the exploratory analysis using RStudio. Here’s an overview of this setup:The interface running the RStudio server could be running on either a Cloud Dataproc master node, a Google Compute Engine virtual machine or even somewhere outside of GCP entirely. One of the advantages of using GCP is that you only need to pay for the RStudio server while it is in use. You can create an RStudio server and shut it off when you’re not using it. This pay-as-you go system also applies to RStudio Pro, the commercial distribution of RStudio.The computation engines available in GCP is where using R in the cloud can really expand your statistical capabilities.The bigrquery package makes it easy to work with data stored in BigQuery by allowing you to query BigQuery tables and retrieve metadata about your projects, datasets, tables, and jobs. The SparkR package, when run on Cloud Dataproc, makes it possible to use R to analyze and structure the data stored in Cloud Storage.Once you have explored and prepared your data for modeling, you can use TensorFlow, Keras and Spark MLlib libraries. You can use the R interface to Tensorflow to work with the high-level Keras and Estimator APIs, and when you need more control, it provides full access to the core TensorFlow API. SparkR jobs on Dataproc allow you to train and score Spark MLlib models at scale. To train and host TensorFlow and Keras models at scale, you can use the R interface to Cloud Machine Learning (ML) Engine and let GCP manage the resources for you.GCP offers an end-to-end managed environment for building, training and scoring models in R. Here’s how to connect all these services in your R environment.Using RStudio: A familiar friendThe first step for most R developers is finding a way to use RStudio. While connecting to the cloud from your desktop is an option, many users prefer to have a cloud-based server version of RStudio. This helps you pick up your desktop settings from wherever you’re working, keep a backup of your work outside your personal computer, and put RStudio on the same network as your data sources. Taking advantage of Google’s high-performance network is something that can greatly improve your R performance when you move data into R’s memory space from other parts of the cloud.  If you plan to use Cloud Dataproc, you can follow this tutorialto install the open source version of RStudio server on Cloud Dataproc’s master node and access the web UI security over an SSH SOCKS tunnel. An advantage to running RStudio on Cloud Dataproc is that you can take advantage of Cloud Dataproc Autoscaling (currently in alpha). With autoscaling, you can have a minimum cluster size as you are developing your SparkR logic. Once you submit your job for large-scale processing, you don’t need to do anything different or worry about modifying your server. You simply submit your SparkR job to RStudio, and the Dataproc cluster scales to meet the needs of your job within the intervals that you set.RStudio Pro, the commercial distribution of RStudio, has a few advantages over the open source version. RStudio Server Pro provides features such as team productivity, security, centralized management, metrics, and commercial support directly from RStudio.You can even launch a production-ready deployment of RStudio Pro directly from the Google Cloud Marketplace in a matter of minutes. The instructions in the next section provide guidance on how to deploy RStudio Pro.Launch an instance of RStudio Server Pro from Cloud Launcher (optional)Cloud Marketplace contains hundreds of development stacks, solutions, and services optimized to run on GCP via one-click deployment.To launch RStudio Pro, you use the search bar at the top of the screen:Type RStudio in the Cloud Launcher search bar.Click RStudio Server Pro.Click Launch on Compute Engine.Follow the instructions. When successful, you should see a screen similar to the one below.A quick note on pricing: The page for launching an instance of RStudio Server Pro includes an estimated costs table. To the left, it also says: “Estimated costs are based on 30-day, 24 hours per day usage in Central US region. Sustained use discount is included.” However, note that most individual users use their instances intermittently and shouldn’t expect the estimated costs provided in that table. GCP has a pay-as-you-go pricing model calculated on a per-second usage basis.Submitting SparkR jobs from Cloud DataprocOnce you’ve connected R to Google Cloud, you can submit SparkR jobs using the new beta release in Cloud Dataproc. Here’s how to do that:1. Click Jobs on the left-hand side of the Dataproc user interface.2. Choose a job type of “SparkR” from the Job Type drop-down menu.3. Populate the name of an R file you want to run that is stored either on the cluster or in Cloud Storage (capital R filename required).SparkR jobs can also be used in either the gcloud betaor API. This makes it easy to automate SparkR processing steps so you can retrain models on complete datasets, pre-process data that you send to Cloud ML Engine, or generate tables for BigQuery from unstructured datasets in Cloud Storage.  When submitting a SparkR job, be sure to require the SparkR package and start the SparkR session at the top of your script.An example job output is displayed here:Connecting R to the rest of GCPAs mentioned earlier, the R community has created many packages for interacting with GCP. The following instructions can serve as a cheat sheet for quickly getting up and running with the most common packages and tools used by R developers on GCP, including SparkR.Prerequisite Python dependencies for TensorFlow stack for RStudioTensorFlow for R has two low-level Python dependencies that are difficult to install from within RStudio. If you are using RStudio Server Pro, you can SSH into the machine and run:At the time of this writing, Tensorflow is not supported directly on Cloud Dataproc due to incompatibility of operating system support.  Install additional R packagesTo generate an R session with the GCP integrations described above, run the following command in the R console:This step takes a few minutes. TensorFlow and Keras also restart your R session during installation.After installation is complete, execute the following command to confirm that the installation was successful:You can also (optionally) try out a quick check on your BigQuery by querying a public dataset:Define the connection to BigQuery from R:Then upload and collect jobs:And look at query results:Install the cloudml packageThe cloudml package is a high-level interface between R and Cloud ML Engine, Google’s platform for building massive machine learning models and bringing them to production. Cloud ML Engine provides a seamless interface for accessing cloud resources, letting you focus on model development, data processing, and inference. To get started with the cloudml package:1. Install the rstudio/cloudml package from GitHub with devtools:2. After cloudml is installed, install the Google Cloud SDK (gcloud) to interact with your Google Cloud account from within R.When you run gcloud_install, a few questions appear.3. Press ENTER to use the default values of the questions:Do you want to help improve the Google Cloud SDK (Y/N)?Do you want to continue (Y/N)?Enter a path to an rc file to update, or leave blank to use [/home/rstudio-user/.bashrc]:4. When asked which account you want to use to perform operations for this configuration, select the first account in the project (the default service account).5. Enter the Google Cloud project ID.Using R in the cloudThis is just the start of the ways we have seen customers delving into GCP and the R ecosystem. The architecture diagram below shows an example of a full deployment of R on Google Cloud Platform for building deep learning models.To learn more about this architecture and how to get started with R on Google Cloud, check out this Next ’18 session on Getting Started with Deep Learning Models in R using Google Cloud and RStudio.For more on getting started with SparkR, check out this post or the Apache SparkR documentation.Thanks to additional contributor Michael Quinn, quantitative analyst at Google. Read more »
  • AI in depth: profiling the model training process for TensorFlow on Cloud ML Engine
    If you've usedCloud Machine Learning (ML) Engine, you know that it can train and deploy any TensorFlow, scikit-learn, and XGBoost models at large scale in the cloud. But did you know that Cloud ML Engine also allows you to use TensorFlow’s profiling mechanisms that can help you analyze and improve your model's performance even further?Whether you use low-level APIs such as tf.Graph and tf.Session or high-level APIs such astf.Estimator andtf.Dataset, it can sometimes be useful to understand how models perform at a lower level to tune your code for efficiency. For example, you might be interested in details about model architectures (e.g., device placement and tensor shapes) or about the performance of specific batch steps (e.g., execution time, memory consumption, or expensive operations).In this post, we show you different tools that can help you gain useful insights into your Cloud ML Engine profiling information so that you can squeeze out that extra bit of performance for your models.The examples presented in this post are based on this codelab and this notebook, which analyze a US natality dataset to predict newborns’ birth weights. While not necessary, you can follow the codelab first to familiarize yourself with the code. You can find the full code samples for this post and their prerequisites here.Basic profilingThe simplest tool at your disposal for profiling the model training process is tf.train.ProfilerHook. ProfilerHook captures profile traces for individual steps that can give you an overview of the individual TensorFlow operations (i.e., the low-level API calls associated with the nodes in your TensorFlow graph), their dependencies and how they are attributed to hardware devices (CPUs, GPUs, and TPUs). In turn, `ProfilerHook` can help you identify possible bottlenecks so you can make targeted improvements to your pipeline and choose the right Cloud ML Engine cluster configuration.If you already use TensorBoard to visualize your TensorFlow graph and store the profile traces in the same directory as the one used for your checkpoints, you will see two additional tabs named “Memory” and “Compute Time” in TensorBoard, at the bottom of the right sidebar. You will also see information about the total compute time, memory size, and tensor output shapes when clicking on a node, as described here.Capturing traces for every step over the entire training process is often impractical, because that process can become resource-intensive, significantly increase your training times, and generate volumes of information that are too large to analyze. To reduce the volume of generated information, you can lower the sampling rate by using either the save_steps or the save_secs attributes to only save profile traces respectively every N steps or N seconds. Below is an example that captures traces every 10 steps:If you want more control over which steps you’d like to profile, you can provide your own implementation of the step selection logic. For example, you can refer to LambdaProfilerHook, a custom extension of ProfilerHook that allows you to select arbitrary steps. One way to use it is to select a range of consecutive steps to profile a specific stage of the training process (in this case steps 100 to 110):ProfilerHook generates one separate trace file named timeline-[STEP_NUMBER].json in the trace-event format for each selected step. To analyze the information contained in those files, first download them locally:Then open the Chrome browser and type chrome://tracing in the URL bar. Then click the Load button and select a trace file. This loads the traces for the corresponding step into the Chrome tracing UI.In the next sections, we show you how to collect and analyze profile traces for different Cloud ML Engine scale tiers (a single machine, a distributed cluster, and a single GPU instance).Example on a single machineLet’s first take a look at some traces captured for our sample training process using the BASIC scale tier, that is, with a single worker instance with four CPUs:The above command runs the training job in 50K batch steps and generates traces every 20K steps, generating three profile trace files:After downloading and loading one of the generated files in the Chrome tracing UI, you can see the execution time and dependencies for every operation in the graph:Traces are divided in three main groups:Compute: Visualizes when each operation started and ended and in what thread it was executed. By clicking on an operation you can reveal more details about its execution times. By clicking on View options > Flow events you can reveal the operations’  dependencies.Tensors: Visualizes when each tensor was created and deleted from memory. The circles represent the exact times at which the memory snapshots were taken for each tensor. By clicking on a tensor you can get more details about its shape, memory consumption, and creation and deletion times.Allocators: Visualizes the overall memory consumption while the batch step was processed. By clicking on the barplot you can reveal the exact memory consumption values at various points in time.By inspecting the graphs, you can see that the speed or performance of your training job is mainly limited by the data input pipeline (since it’s the longest one). To implement some optimizations, you might want to check out the documentation on data input pipeline performance.Example on a distributed clusterLet’s now take a look at the same example but this time running the training job on the STANDARD_1 scale tier, i.e., one master node with four CPUs and four worker instances with eight CPUs each:The above command generates a total of 15 trace files (three for the master and three for each of the four workers). Besides the usual metrics, the workers’ trace files also contain the parameter servers’ operations. (Note: the parameter server stores and updates the model's parameters in a distributed architecture.)If you open one of the worker’s trace files, you see that the traces divided in the same types of groups as in the first example (Compute, Tensors, and Allocators). However there are now Compute and Tensors groups for all workers (e.g., a Compute group is named /job:/worker/replica:0/task:N/device:CPU:0 Compute for an Nth worker) and for all parameter servers (e.g., a Tensors group is named /job:/ps/replica:0/task:N/device:CPU:0 Tensors for the Nth parameter server).By inspecting the graphs, you clearly see that there is some communication overhead between the nodes (that is, time spent on calls to RecvTensor):This comparison highlights the tradeoff inherent in distributed architectures: using more nodes can help the training process reach convergence quicker but also consumes more computational resources. It’s important to measure actual consumption so that you can finely adjust the number of workers you’re using, in order to reach the ideal balance between speed and cost. Capturing and visualizing traces can help you estimate your overhead, letting you more precisely select the architecture that is suited to your use case.A GPU exampleLet’s now take a look again at the same example but this time running the training job on the BASIC_GPU scale tier, that is, a single worker instance with eight CPUs and one GPU:If you open one of the generated trace files, you see that things look a bit different from our previous examples:There still is an Allocators group for the memory consumption, however multiple allocators are displayed: CPU, cpu_pool, GPU and cuda_host_buffer. You can read more about cuda_host here.Statistics about the operations run in every GPU stream (including the memcpy stream) are available in each one of the Compute groups. You can find more details about CUDA streams here or here.By inspecting the graphs, you see that the GPU utilization is quite low. This means either that GPUs are not the right fit for this model, or that the batch size must be increased significantly. You also see that the input data pipeline takes up a large share of the overall time spent for the whole step:Advanced profilingIn cases where collecting basic traces for individual steps isn’t enough, TensorFlow offers another great tool called Profiler that lets you do some advanced analysis. Profiler allows you to aggregate traces for multiple steps and calculate average values for execution times, CPU and memory consumption. It also allows you to search for the most time- or resource-expensive operations, to analyze device placement and model architecture (number of parameters, tensor shapes), and more.To use Profiler, simply instantiate a ProfileContext as shown in the following excerpt from our example code:The above code generates a single Profiler context file that contains traces for 50 consecutive steps (that is between steps #1050 and #1100). If you want more control over how Profiler traces are collected, you can customize your profiling options in the ProfilerOptionBuilder class.In the next sections, we show you two different ways to use Profiler traces: using the command line and using the standard user interface.Using Profiler with the command lineFollow these steps to use Profiler with the command line:1. Install the Bazel build tool by following the instructions for your platform. (Note: if you’ve already compiled TensorFlow from source, you should already have Bazel installed.)2. Download and compile Profiler using Bazel:3. Download the Profiler’s trace file locally:4. Start a new Profiler session:Now let’s take a look at a few examples of what you can do in Profiler by passing different options to the command line tool:1. Display the python methods invocation tree:The above command displays average statistics for the 50 profiled steps. You can also specify the exact steps that you are interested in by using the `regExes` filters.2. Expose the operations that consume the most memory:3. Display the tensor shapes and the number of parameters for all calls made to tf.trainabale_variables() in the profiled training steps:Lastly, you can also experiment with the advise command to automatically profile your model and easily find the most expensive operations, the accelerators’ utilization, and more.You can also visualize a timeline for a specific step to inspect with chrome://tracing as we’ve discussed above (read more about how to do this), or generate your own timelines for a few steps. First,  generate a binary profile (taking into account your desired conditions) and then generate a visualization with pprof (if you’ve installed it):tfprof> code -select accelerator_micros -max_depth 100000 -output pprof:outfile=PATH_TO_YOUR_FILE  -trim_name_regexes .*apply_op.*And now execute this command from your terminal to generate a png file:Using the Profiler UIThere’s also a user interface tool called profiler-ui to analyze your Profiler trace files. To use profiler-ui, first follow the installation instructions, then start the tool:This tool has the same capabilities as the command line interface but it is generally easier to use and can help you visually identify bottlenecks in the flow of operations.ConclusionWe hope you find these techniques useful when training models, to better understand the performance and behavior of your training process. If you’re interested in learning more, check out some of our documentation below:Cloud ML Engine documentationTensorFlow guide to collecting runtime statisticsTensorFlow Profiler documentationTensorFlow CLI profiling tool optionsTraceEvent format for TensorFlowChromium Trace event profiling toolTensorflow Model Benchmark tool for inference profiling Read more »
WordPress RSS Feed Retriever by Theme Mason

AI – Latest News

WordPress RSS Feed Retriever by Theme Mason

ScienceDaily – Artificial Intelligence News

  • Robots track moving objects with unprecedented precision
    A novel system uses RFID tags to help robots home in on moving objects with unprecedented speed and accuracy. The system could enable greater collaboration and precision by robots working on packaging and assembly, and by swarms of drones carrying out search-and-rescue missions. Read more »
  • Artificial intelligence to boost Earth system science
    A new study shows that artificial intelligence can substantially improve our understanding of the climate and the Earth system. Read more »
  • The first walking robot that moves without GPS
    Desert ants are extraordinary solitary navigators. Researchers were inspired by these ants as they designed AntBot, the first walking robot that can explore its environment randomly and go home automatically, without GPS or mapping. This work opens up new strategies for navigation in autonomous vehicles and robotics. Read more »
  • Getting a grip on human-robot cooperation
    There is a time when a successful cooperation between humans and robots has decisive importance: it is in the precise moment that one "actor" is required to hand an object to another "actor" and, therefore, to coordinate their actions accordingly. But how can we make this interaction more natural for robots? Read more »
  • Teaching self-driving cars to predict pedestrian movement
    By zeroing in on humans' gait, body symmetry and foot placement, researchers are teaching self-driving cars to recognize and predict pedestrian movements with greater precision than current technologies. Read more »
  • Toward automated animal identification in wildlife research
    A new program automatically detects regions of interest within images, alleviating a serious bottleneck in processing photos for wildlife research. Read more »
  • Psychology: Robot saved, people take the hit
    To what extent are people prepared to show consideration for robots? A new study suggests that, under certain circumstances, some people are willing to endanger human lives -- out of concern for robots. Read more »
  • Citizen science projects have a surprising new partner, the computer
    Data scientists and citizen science experts partnered with ecologists who often study wildlife populations by deploying camera traps. These camera traps are remote, independent devices, triggered by motion and infrared sensors that provide researchers with images of passing animals. The researchers built skill sets to help computers identify other animals, such as a deer or squirrel, with even fewer images. Read more »
  • Walking with Pokémon
    In a recent study, researchers reveal how the Pokémon GO augmented reality game positively impact the physical activity in players over 40. The authors hope the findings will inform urban planners and game designers to inspire people to be more active. Read more »
  • Robot combines vision and touch to learn the game of Jenga
    Machine-learning approach could help robots assemble cellphones and other small parts in a manufacturing line. Read more »
  • Atari master: New AI smashes Google DeepMind in video game challenge
    A new breed of algorithms has mastered Atari video games 10 times faster than state-of-the-art AI, with a breakthrough approach to problem solving. Read more »
  • Most people overlook artificial intelligence despite flawless advice
    A team of researchers recently discovered that most people overlook artificial intelligence despite flawless advice. AI-like systems will be an integral part of the Army's strategy over the next five years, so system designers will need to start getting a bit more creative in order to appeal to users. Read more »
  • Engineers translate brain signals directly into speech
    In a scientific first, neuroengineers have created a system that translates thought into intelligible, recognizable speech. This breakthrough, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain. Read more »
  • Defending against cyberattacks by giving attackers 'false hope'
    'The quarantine is a decoy that behaves very similar to the real compromised target to keep the attacker assuming that the attack is still succeeding. In a typical cyberattack the more deeply attackers go in the system, the more they have the ability to go many directions. It becomes like a Whack-A-Mole game for those defending the system. Our strategy simply changes the game, but makes the attackers think they are being successful.' Read more »
  • Self-driving cars, robots: Identifying AI 'blind spots'
    A novel model identifies instances in which autonomous systems have 'learned' from training examples that don't match what's actually happening in the real world. Engineers could use this model to improve the safety of artificial intelligence systems, such as driverless vehicles and autonomous robots. Read more »
  • The first tendril-like soft robot able to climb
    Researchers have made the first soft robot mimicking plant tendrils: it is able to curl and climb, using the same physical principles determining water transport in plants. In the future this tendril-like soft robot could inspire the development of wearable devices, such as soft braces, able to actively morph their shape. Read more »
  • Increasing skepticism against robots
    In Europe, people are more reserved regarding robots than they were five years ago. Read more »
  • Artificial intelligence can dramatically cut time needed to process abnormal chest X-rays
    New research has found that a novel Artificial Intelligence (AI) system can dramatically reduce the time needed to ensure that abnormal chest X-rays with critical findings will receive an expert radiologist opinion sooner, cutting the average delay from 11 days to less than three days. Chest X-rays are routinely performed to diagnose and monitor a wide range of conditions affecting the lungs, heart, bones, and soft tissues. Read more »
  • Smart microrobots that can adapt to their surroundings
    Scientists have developed tiny elastic robots that can change shape depending on their surroundings. Modeled after bacteria and fully biocompatible, these robots optimize their movements so as to get to hard-to-reach areas of the human body. They stand to revolutionize targeted drug delivery. Read more »
  • Measuring ability of artificial intelligence to learn is difficult
    Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket, a study has found. Read more »
  • 'Ambidextrous' robots could dramatically speed e-commerce
    Engineers present a novel, 'ambidextrous' approach to grasping a diverse range of object shapes without training. Read more »
  • Smart home tests first elder care robot
    Researchers believe the robot, nicknamed RAS, could eventually help those with dementia and other limitations continue to live independently in their own homes. Read more »
  • Artificial bug eyes
    Single lens eyes, like those in humans and many other animals, can create sharp images, but the compound eyes of insects and crustaceans have an edge when it comes to peripheral vision, light sensitivity and motion detection. That's why scientists are developing artificial compound eyes to give sight to autonomous vehicles and robots, among other applications. Now, a new report describes the preparation of bioinspired artificial compound eyes using a simple low-cost approach. Read more »
  • Can artificial intelligence tell a teapot from a golf ball?
    How smart is the form of artificial intelligence known as deep learning computer networks, and how closely do these machines mimic the human brain? They have improved greatly in recent years, but still have a long way to go, according to a team of cognitive psychologists. Read more »
  • How game theory can bring humans and robots closer together
    Researchers have for the first time used game theory to enable robots to assist humans in a safe and versatile manner. Read more »
  • Bees can count with small number of nerve cells in their brains, research suggests
    Bees can solve seemingly clever counting tasks with very small numbers of nerve cells in their brains, according to researchers. Read more »
  • New AI computer vision system mimics how humans visualize and identify objects
    Researchers have demonstrated a computer system that can discover and identify the real-world objects it 'sees' based on the same method of visual learning that humans use. Read more »
  • Robots with sticky feet can climb up, down, and all around
    Researchers have created a micro-robot whose electroadhesive foot pads, inspired by the pads on a gecko's feet, allow it to climb on vertical and upside-down conductive surfaces, like the inside walls of a commercial jet engine. Groups of them could one day be used to inspect complicated machinery and detect safety issues sooner, while reducing maintenance costs. Read more »
  • Computer hardware designed for 3D games could hold the key to replicating human brain
    Researchers have created the fastest and most energy efficient simulation of part of a rat brain using off-the-shelf computer hardware. Read more »
  • Computer chip vulnerabilities discovered
    A research team has uncovered significant and previously unknown vulnerabilities in high-performance computer chips that could lead to failures in modern electronics. Read more »
  • New models sense human trust in smart machines
    New 'classification models' sense how well humans trust intelligent machines they collaborate with, a step toward improving the quality of interactions and teamwork. Read more »
  • Mountain splendor? Scientists know where your eyes will look
    Using precise brain measurements, researchers predicted how people's eyes move when viewing natural scenes, an advance in understanding the human visual system that can improve a host of artificial intelligence efforts, such as the development of driverless cars. Read more »
  • Computers successfully trained to identify animals in photos
    Researchers trained a deep neural network to classify wildlife species using 3.37 million camera-trap images of 27 species of animals obtained from five states across the United States. The model then was tested on nearly 375,000 animal images at a rate of about 2,000 images per minute on a laptop computer, achieving 97.6 percent accuracy -- likely the highest accuracy to date in using machine learning for wildlife image classification. Read more »
  • Smarter AI: Machine learning without negative data
    A research team has successfully developed a new method for machine learning that allows an AI to make classifications without what is known as 'negative data,' a finding which could lead to wider application to a variety of classification tasks. Read more »
  • Aquatic animals that jump out of water inspire leaping robots
    Ever watch aquatic animals jump out of the water and wonder how they manage to do it in such a streamlined and graceful way? Researchers who specialize in water entry and exit in nature had the same question. Read more »
  • Model of quantum artificial life on quantum computer
    Researchers have developed a quantum biomimetic protocol that reproduces the characteristic process of Darwinian evolution adapted to the language of quantum algorithms and quantum computing. The researchers anticipate a future in which machine learning, artificial intelligence and artificial life itself will be combined on a quantum scale. Read more »
  • Android child's face strikingly expressive
    Android faces must express greater emotion if robots are to interact with humans more effectively. Researchers tackled this challenge as they upgraded their android child head, named Affetto. They precisely examined Affetto's facial surface points and the precise balancing of different forces necessary to achieve more human-like motion. Through mechanical measurements and mathematical modeling, they were able to use their findings to greatly enhance Affetto's range of emotional expression. Read more »
  • AI capable of outlining in a single chart information from thousands of scientific papers
    Scientists have developed a Computer-Aided Material Design (CAMaD) system capable of extracting information related to fabrication processes and material structures and properties -- factors vital to material design -- and organizing and visualizing the relationship between them. The use of this system enables information from thousands of scientific and technical articles to be summarized in a single chart, rationalizing and expediting material design. Read more »
  • Artificial intelligence may fall short when analyzing data across multiple health systems
    A new study shows deep learning models must be carefully tested across multiple environments before being put into clinical practice. Read more »
  • Codebreaker Turing's theory explains how shark scales are patterned
    A system proposed by world war two codebreaker Alan Turing more than 60 years ago can explain the patterning of tooth-like scales possessed by sharks, according to new research. Read more »
  • Could machines using artificial intelligence make doctors obsolete?
    The technology of these tools is evolving rapidly. Standalone machines can now perform limited tasks raising the question of whether machines will ever completely replace doctors? Read more »
  • New method peeks inside the 'black box' of artificial intelligence
    Computer scientists have developed a promising new approach for interpreting machine learning algorithms. Unlike previous efforts, which typically sought to 'break' the algorithms by removing key words from inputs to yield the wrong answer, the researchers instead reduced the inputs to the bare minimum required to yield the correct answer. On average, the researchers got the correct answer with an input of less than three words. Read more »
  • Shape-shifting robots perceive surroundings, make decisions for first time
    Researchers have developed modular robots that can perceive their surroundings, make decisions and autonomously assume different shapes in order to perform various tasks -- an accomplishment that brings the vision of adaptive, multipurpose robots a step closer to reality. Read more »
  • Empathetic machines favored by skeptics but might creep out believers
    Most people would appreciate a chatbot that offers sympathetic or empathetic responses, according to a team of researchers, but they added that reaction may rely on how comfortable the person is with the idea of a feeling machine. Read more »
  • Machines that learn language more like kids do
    Researchers describe a parser that learns through observation to more closely mimic a child's language-acquisition process, which could greatly extend the parser's capabilities. Read more »
  • Humans help robots learn tasks
    With a smartphone and a browser, people worldwide will be able to interact with a robot to speed the process of teaching robots how to do basic tasks. Read more »
  • Shielded quantum bits
    A theoretical concept to realize quantum information processing has been developed by a team of physicists. Read more »
  • Artificial fly brain can tell who's who
    Researchers have built a neural network that mimics the fruit fly's visual system and can distinguish and re-identify flies. This provides evidence that the humble fruit fly's vision is clearer than previously thought. Read more »
  • Artificial intelligence controls quantum computers
    Researchers present a quantum error correction system that is capable of learning thanks to artificial intelligence. Read more »
  • New tool streamlines the creation of moving pictures
    It's often easy to imagine balloons soaring or butterflies fluttering across a still image, but realizing this vision through computer animation is easier said than done. Now, a team of researchers has developed a new tool that makes animating such images much simpler. Read more »
  • Where deep learning meets metamaterials
    A new study uses 'deep-learning' computer networks inspired by the layered and hierarchical architecture of the human brain to design basic nanophotonic, metamaterial elements for energy harvesting and medical diagnostics. Read more »
  • Clapping Music app reveals that changing rhythm isn't so easy
    Scientists have developed an app to understand why some rhythms are more difficult to perform than others. Read more »
  • New reservoir computer marks first-ever microelectromechanical neural network application
    A group of researchers reports the construction of the first reservoir computing device built with a microelectromechanical system. The neural network exploits the nonlinear dynamics of a microscale silicon beam to perform its calculations. The group's work looks to create devices that can act simultaneously as a sensor and a computer using a fraction of the energy a normal computer would use. Read more »
  • Artificial intelligence helps reveal how people process abstract thought
    As artificial intelligence becomes more sophisticated, much of the public attention has focused on how successfully these technologies can compete against humans at chess and other strategy games. A philosopher has taken a different approach, deconstructing the complex neural networks used in machine learning to shed light on how humans process abstract learning. Read more »
  • Model helps robots navigate more like humans do
    Researchers have now devised a way to help robots navigate environments more like humans do. Their novel motion-planning model lets robots determine how to reach a goal by exploring the environment, observing other agents, and exploiting what they've learned before in similar situations. Read more »
  • A new brain-inspired architecture could improve how computers handle data and advance AI
    Researchers are developing a new computer architecture, better equipped to handle increased data loads from artificial intelligence. Their designs draw on concepts from the human brain and significantly outperform conventional computers in comparative studies. Read more »
  • Robot masters human balancing act
    By translating a key human physical skill, whole-body balance, into an equation, engineers used the numerical formula to program their robot Mercury. Read more »
  • Smart technology for synchronized 3D printing of concrete
    Scientists have developed a technology where two robots can work in unison to 3D-print a concrete structure. Read more »
  • Tiny soft robot with multilegs paves way for drugs delivery in human body
    A novel tiny, soft robot with caterpillar-like legs capable of carrying heavy loads and adaptable to adverse environment has just been developed. This mini delivery-robot could pave way for medical technology advancement such as drugs delivery in human body. Read more »
  • Robots may need lizard-like tails for 'off-road' travel
    Robots may one day tackle obstacles and traverse uneven terrains thanks to collaborative research analyzing the motion of lizards. The study used a slow motion camera to capture the nuanced movement of eight species of Australian agamid lizards that run on two legs -- an action known as 'bipedal' movement. Read more »
WordPress RSS Feed Retriever by Theme Mason

AI Trends – AI News and Events

  • Why AI Has Yet to Reshape Most Businesses
    The art of making perfumes and colognes hasn’t changed much since the 1880s, when synthetic ingredients began to be used. Expert fragrance creators tinker with combinations of chemicals in hopes of producing compelling new scents. So Achim Daub, an executive at one of the world’s biggest makers of fragrances, Symrise, wondered what would happen if he injected artificial intelligence into the process. Would a machine suggest appealing formulas that a human might not think to try? Daub hired IBM to design a computer system that would pore over massive amounts of information—the formulas of existing fragrances, consumer data, regulatory information, on and on—and then suggest new formulations for particular markets. The system is called Philyra, after the Greek goddess of fragrance. Evocative name aside, it can’t smell a thing, so it can’t replace human perfumers. But it gives them a head start on creating something novel. Daub is pleased with progress so far. Two fragrances aimed at young customers in Brazil are due to go on sale there in June. Only a few of the company’s 70 fragrance designers have been using the system, but Daub expects to eventually roll it out to all of them. However, he’s careful to point out that getting this far took nearly two years—and it required investments that still will take a while to recoup. Philyra’s initial suggestions were horrible: it kept suggesting shampoo recipes. After all, it looked at sales data, and shampoo far outsells perfume and cologne. Getting it on track took a lot of training by Symrise’s perfumers. Plus, the company is still wrestling with costly IT upgrades that have been necessary to pump data into Philyra from disparate record-­keeping systems while keeping some of the information confidential from the perfumers themselves. “It’s kind of a steep learning curve,” Daub says. “We are nowhere near having AI firmly and completely established in our enterprise system.” The perfume business is hardly the only one to adopt machine learning without seeing rapid change. Despite what you might hear about AI sweeping the world, people in a wide range of industries say the technology is tricky to deploy. It can be costly. And the initial payoff is often modest. It’s one thing to see breakthroughs in artificial intelligence that can outplay grandmasters of Go, or even to have devices that turn on music at your command. It’s another thing to use AI to make more than incremental changes in businesses that aren’t inherently digital. AI might eventually transform the economy—by making new products and new business models possible, by predicting things humans couldn’t have foreseen, and by relieving employees of drudgery. But that could take longer than hoped or feared, depending on where you sit. Most companies aren’t generating substantially more output from the hours their employees are putting in. Such productivity gains are largest at the biggest and richest companies, which can afford to spend heavily on the talent and technology infrastructure necessary to make AI work well. This doesn’t necessarily mean that AI is overhyped. It’s just that when it comes to reshaping how business gets done, pattern-recognition algorithms are a small part of what matters. Far more important are organizational elements that ripple from the IT department all the way to the front lines of a business. Pretty much everyone has to be attuned to how AI works and where its blind spots are, especially the people who will be expected to trust its judgments. All this requires not just money but also patience, meticulousness, and other quintessentially human skills that too often are in short supply. Looking for unicorns Last September, a data scientist named Peter Skomoroch tweeted: “As a rule of thumb, you can expect the transition of your enterprise company to machine learning will be about 100x harder than your transition to mobile.” It had the ring of a joke, but Skomoroch wasn’t kidding. Several people told him they were relieved to hear that their companies weren’t alone in their struggles. “I think there’s a lot of pain out there—inflated expectations,” says Skomoroch, who is CEO of SkipFlag, a business that says it can turn a company’s internal communications into a knowledge base for employees. “AI and machine learning are seen as magic fairy dust.” Among the biggest obstacles is getting disparate record-keeping systems to talk to each other. That’s a problem Richard Zane has encountered as the chief innovation officer at UC Health, a network of hospitals and medical clinics in Colorado, Wyoming, and Nebraska. It recently rolled out a conversational software agent called Livi, which uses natural-­language technology from a startup called Avaamo to assist patients who call UC Health or use the website. Livi directs them to renew their prescriptions, books and confirms their appointments, and shows them information about their conditions. Zane is pleased that with Livi handling routine queries, UC Health’s staff can spend more time helping patients with complicated issues. But he acknowledges that this virtual assistant does little of what AI might eventually do in his organization. “It’s just the tip of the iceberg, or whatever the positive version of that is,” Zane says. It took a year and a half to deploy Livi, largely because of the IT headaches involved with linking the software to patient medical records, insurance-billing data, and other hospital systems. Similar setups bedevil other industries, too. Some big retailers, for instance, save supply-chain records and consumer transactions in separate systems, neither of which is connected to broader data storehouses. If companies don’t stop and build connections between such systems, then machine learning will work on just some of their data. That explains why the most common uses of AI so far have involved business processes that are siloed but nonetheless have abundant data, such as computer security or fraud detection at banks. Even if a company gets data flowing from many sources, it takes lots of experimentation and oversight to be sure that the information is accurate and meaningful. When Genpact, an IT services company, helps businesses launch what they consider AI projects, “10% of the work is AI,” says Sanjay Srivastava, the chief digital officer. “Ninety percent of the work is actually data extraction, cleansing, normalizing, wrangling.” Those steps might look seamless for Google, Netflix, Amazon, or Facebook. But those companies exist to capture and use digital data. They’re also luxuriously staffed with PhDs in data science, computer science, and related fields. “That’s different than the rank and file of most enterprise companies,” Skomoroch says. Indeed, smaller companies often require employees to delve into several technical domains, says Anna Drummond, a data scientist at Sanchez Oil and Gas, an energy company based in Houston. Sanchez recently began streaming and analyzing production data from wells in real time. It didn’t build the capability from scratch: it bought the software from a company called MapR. But Drummond and her colleagues still had to ensure that data from the field was in formats a computer could parse. Drummond’s team also got involved in designing the software that would feed information to engineers’ screens. People adept at all those things are “not easy to find,” she says. “It’s like unicorns, basically. That’s what’s slowing down AI or machine-learning adoption.” Fluor, a huge engineering company, spent about four years working with IBM to develop an artificial-intelligence system to monitor massive construction projects that can cost billions of dollars and involve thousands of workers. The system inhales both numeric and natural-language data and alerts Fluor’s project managers about problems that might later cause delays or cost overruns. Data scientists at IBM and Fluor didn’t need long to mock up algorithms the system would use, says Leslie Lindgren, Fluor’s vice president of information management. What took much more time was refining the technology with the close participation of Fluor employees who would use the system. In order for them to trust its judgments, they needed to have input into how it would work, and they had to carefully validate its results, Lindgren says. To develop a system like this, “you have to bring your domain experts from the business—I mean your best people,” she says. “That means you have to pull them off other things.” Using top people was essential, she adds, because building the AI engine was “too important, too long, and too expensive” for them to do otherwise. Read the source article at MIT Technology Review. Read more »
  • DOD Unveils Its AI Strategy Following White House Executive Order
    The Defense Department launched its artificial intelligence strategy on Feb. 12 in concert with the White House executive order that created the American Artificial Intelligence Strategy. “The [executive order] is paramount for our country to remain a leader in AI, and it will not only increase the prosperity of our nation, but also enhance our national security,” Dana Deasy, DOD’s chief information officer, said in a media roundtable today. The CIO and Air Force Lt. Gen. Jack Shanahan, first director of DOD’s Joint Artificial Intelligence Center, discussed the strategy’s launch with reporters. The National Defense Strategy recognizes that the U.S. global landscape has evolved rapidly, with Russia and China making significant investments to modernize their forces, Deasy said. “That includes substantial funding for AI capabilities,” he added. “The DOD AI strategy directly supports every aspect of the NDS.” As stated in the AI strategy, he said, the United States — together with its allied partners — must adopt AI to maintain its strategic position to prevail on future battlefields and safeguard a free and open international order. Speed and Agility Are Key Increasing speed and agility is a central focus on the AI strategy, the CIO said, adding that those factors will be delivered to all DOD AI capabilities across every DOD mission. “The success of our AI initiatives will rely upon robust relationships with internal and external partners. Interagency, industry, our allies and the academic community will all play a vital role in executing our AI strategy,” Deasy said. “I cannot stress enough the importance that the academic community will have for the JAIC,” he noted. “Young, bright minds continue to bring fresh ideas to the table, looking at the problem set through different lenses. Our future success not only as a department, but as a country, depends on tapping into these young minds and capturing their imagination and interest in pursuing the job within the department.” Reforming DOD Business The last part of the NDS focuses on reform, the CIO said, and the JAIC will spark many new opportunities to reform the department’s business processes. “Smart automation is just one such area that promises to improve both effectiveness and efficiency,” he added. AI will use an enterprise cloud foundation, which will also increase efficiencies across DOD, Deasy said. He noted that DOD will emphasize responsibility and use of AI through its guidance and vision principles for using AI in a safe, lawful and ethical way. JAIC: A Focal Point of AI “It’s hard to overstate the importance of operationalizing AI across the department, and to do so with the appropriate sense of urgency and alacrity,” JAIC director Shanahan told reporters. The DOD AI strategy applies to the entire department, he said, adding the JAIC is a focal point of the strategy. The JAIC was established in response to the 2019 National Defense Authorization Act, and stood up in June 2018 “to provide a common vision, mission and focus to drive department-wide AI capability delivery.” Mission Themes The JAIC has several critical mission themes, Shanahan said. — First is the effort to accelerate delivery and adoption of AI capabilities across DOD, he noted. “This underscores the importance of transitioning from research and development to operational-fielded capabilities,” he said. “The JAIC will operate across the full AI application lifecycle, with emphasis on near-term execution and AI adoption.” — Second is to establish a common foundation for scaling AI’s impact, Shanahan said. “One of the JAIC’s most-important contributions over the long term will be establishing a common foundation enabled by enterprise cloud with particular focus on shared data repositories for useable tools, frameworks and standards and cloud … services,” he explained. — Third, to synchronize DOD AI activities, related AI and machine-learning projects are ongoing across the department, and it’s important to ensure alignment with the National Defense Strategy, the director said. — Last is the effort to attract and cultivate a world-class AI team, Shanahan said. Two pilot programs that are national mission initiatives – a  broad, joint cross-cutting AI challenge – comprise preventive maintenance and humanitarian assistance and disaster relief, the director said, adding that “initial capabilities [will be] delivered over the next six months.” Read the source coverage at US Department of Defense. Read more »
  • GoTo Fail and AI Brittleness: The Case of AI Self-Driving Cars
    By Lance Eliot, the AI Trends Insider I’m guessing that you’ve likely heard or read the famous tale of the Dutch boy that plugged a hole in a leaking dam via his finger and was able to save the entire country by doing so. I used to read this fictional story to my children when they were quite young. They delighted in my reading of it, often asking me to read it over and over. One aspect that puzzled my young children was how a hole so small that it could be plugged by a finger could potentially jeopardize the integrity of the entire dam. Rather astute of them to ask. I read them the story to impart a lesson of life that I had myself learned over the years, namely that sometimes the weakest link in the chain can undermine an entire system, and incredibly too the weakest link can be relatively small and surprisingly catastrophic in spite of its size. I guess that’s maybe two lessons rolled into one. The first part is that the weakest link in a chain can become broken or severed and thus the whole chain no longer exists as a continuous chain. By saying it is the weakest link, we’re not necessarily saying its size, and it could be a link of the same size as the rest of the chain. It could be even a larger link or perhaps even the largest link of the chain. Or, it could be a smaller link or possibly the smallest sized link of the chain. The point being that by size alone, it is not of necessity the basis for why the link might be the weakest. There could be a myriad of other reasons why the link is subject to being considered “the weakest” and for which size might or might not particularly matter. Another perhaps obvious corollary regarding the weakest link aspect is that it is just one link involved. That’s what catches our attention and underlies the surprise about the notion. We might not be quite so taken aback if a multitude of links broke and therefore the chain itself came into ruin. The second part of the lesson learned involves the cascading impact and how severe it can be as a consequence of the weakest link giving way. In the case of the tiny hole in the dam, presumably the water could rush through that hole and the build-up of pressure would tend to crack and undermine the dam at that initial weakest point. As the water pushes and pushes to get through it, the finger-sized hole is bound to grow and grow in size, until inextricably the hole becomes a gap, and the gap then becomes a breech, and the breech then leads to the entire dam crumbling and being overtaken by the madly and punishingly flowing water. If you are not convinced that a single weakest link could undermine a much larger overall system, I’d like to enchant you with the now-famous account of the so-called “goto fail goto fail” saga that played out in February 2014. This is a true story. The crux of the story is that one line of code, a single “Go To” statement in a software routine, led to the undermining of a vital aspect of computer security regarding Apple related devices. I assert that the one line of code is the equivalent to a tiny finger-sized hole in a dam. Via that one hole, a torrent of security guffaws could have flowed.  At the time, and still to this day, there were reverberations that this single “Go To” statement could have been so significant. For those outside of the computer field, it seemed shocking. What, one line of code can be that crucial? For those within the computer field, there was for some a sense of embarrassment, namely that the incident laid bare the brittleness of computer programs and software, along with being an eye opener to the nature of software development. I realize that there were pundits that said it was freakish and a one-of-a-kind, but at the time I concurred with those that said this is actually just the tip of the iceberg. Little do most people know or understand how software is often built on a house of cards. Depending upon how much actual care and attention you devote to your software efforts, which can be costly in terms of time, labor, and resources needed, you can make it hard to have a weakest link or you can make it relatively easy to have a weakest link. All told, you cannot assume that all software developers and all software development efforts are undertaking the harder route of trying to either prevent weakest links or at least catch the weakest link when it breaks. As such, as you walk and talk today, and are either interacting with various computer systems or reliant upon those computer systems, you have no immediate way to know whether there is or is not a weakest link ready to be encountered. In the case of the “Go to” line of code that I’m about to show you, it turns out that the inadvertent use of a somewhat errant “Go to” statement created an unreachable part of the program, which is often referred to as an area of code known as dead code. It is dead code because it will never be brought to life, in the sense that it will never be executed during the course of the program being run. Why would you have any dead code in your program? Normally, you would not. A programmer ought to be making sure that their is code is reachable in one manner or another. Having code that is unreachable is essentially unwise since it is sitting in the program but won’t ever do anything. Furthermore, it can be quite confusing to any other programmer that comes along to take a look at the code. There are times at which a programmer might purposely put dead code into their program and have in mind that at some future time they will come back to the code and change things so that the dead code then becomes reachable. It is a placeholder. Another possibility is that the code was earlier being used, and for some reason the programmer decided they no longer wanted it to be executed, so they purposely put it into a spot that it became dead code in the program, or routed the execution around the code so that it would no longer be reachable and thus be dead code. They might for the moment want to keep the code inside the program, just in case they later decide to encompass it again later on. Generally, the dead code is a human programmer consideration in that if a programmer has purposely included dead code it raises questions about why and what it is there for, since it won’t be executed. There is a strong possibility that the programmer goofed-up and didn’t intend to have dead code. Our inspection of the code won’t immediately tell us whether the programmer put the dead code there for a purposeful reason, or they might have accidentally formulated a circumstance of dead code and not even realize they did so. That’s going to be bad because the programmer presumably assumed that the dead code would get executed at some juncture while the program was running, but it won’t. Infamous Dead Code Example You are now ready to see the infamous code (it’s an excerpt, the entire program is available as open source online at many code repositories). Here it is:    if ((err = ReadyHash(&SSLHashSHA1, &hashCtx)) != 0)        goto fail;    if ((err = SSLHashSHA1.update(&hashCtx, &clientRandom)) != 0)        goto fail;    if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0)        goto fail;    if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)        goto fail;        goto fail;    if ((err =, &hashOut)) != 0)        goto fail; err = sslRawVerify(ctx,                    ctx->peerPubKey,                    dataToSign,                /* plaintext */                    dataToSignLen,            /* plaintext length */                    signature,                    signatureLen);    if(err) {     sslErrorLog(“SSLDecodeSignedServerKeyExchange: sslRawVerify “                    “returned %d\n“, (int)err);        goto fail;    } fail: SSLFreeBuffer(&signedHashes); SSLFreeBuffer(&hashCtx);    return err; Observe that there appear to be five IF statements, one after another. Each of the IF statements seems to be somewhat the same, namely each tests a condition and if the condition is true then the code is going to jump to the label of “fail” that is further down in the code. All of this would otherwise not be especially worth discussing, except for the fact that there is a “goto fail” hidden amongst that set of a series of five IF statements. It is actually on its own and not part of any of those IF statements. It is sitting in there, among those IF statements, and will be executed unconditionally, meaning that once it is reached, the program will do as instructed and jump to the label “fail” that appears further down in the code. Can you see the extra “goto fail” that has found its ways into that series of IF statements? It might take a bit of an eagle eye for you to spot it. In case you don’t readily see it, I’ll include the excerpt again here and show you just the few statements I want you to focus on for now:    if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)        goto fail;        goto fail;    if ((err =, &hashOut)) != 0)        goto fail; What you have in a more abstract way is these three statements:    IF (condition) goto fail;    goto fail;    IF (condition) go to fail; There is an IF statement, the first of those above three lines, that has its own indication of jumping to the label “fail” when the assessed condition is true. Immediately after that IF statement, there is a statement that says “goto fail” and it is all on its own, that’s the second line of the three lines. The IF statement that follows that “goto fail” which is on its own, the third line, won’t ever be executed. Why? Because the “goto fail” in front of it will branch away and the sad and lonely IF statement won’t get executed. In fact, all of the lines of code following that “goto fail” are going to be skipped during execution. They are in essence unreachable code. They are dead code. By the indentation, it becomes somewhat harder to discern that the unconditional GO TO statement exists within the sequence of those IF statements. One line of code, a seemingly extraneous GO TO statement, which is placed in a manner that it creates a chunk of unreachable code. This is the weakest link in this chain. And it creates a lot of troubles. By the way, most people tend to refer to this as the “goto fail goto fail” because it has two such statements together. There were T-shirts, bumper stickers, coffee mugs, and the like, all quickly put into the marketplace at the time of this incident, allowing the populace to relish the matter and showcase what it was about. Some of the versions said “goto fail; goto fail;” and included the proper semi-colons while others omitted the semi-colons. What was the overall purpose of this program, you might be wondering? It was an essential part of the software that does security verification for various Apple devices like their smartphones, iPad, etc. You might be aware that when you try to access a web site, there is a kind of handshake that allows a secure connection to be potentially established. The standard used for this is referred to as the SSL/TSL, or the Secure Socket Layer / Transport Security Layer. When your device tries to connect with a web site and SSL/TSL is being used, the device starts to make the connection, the web site presents a cryptographic certificate for verification purposes, and your device then tries to verify that the certificate is genuine (along with other validations that occur). In the excerpt that I’ve shown you, you are looking at the software that would be sitting in your Apple device and trying to undertake that SSL/TSL verification. Unfortunately, regrettably, the dead code is quite important to the act of validating the SSL/TSL certificate and other factors. Essentially, by bypassing an important part of the code, this program is going to be falsely reporting that the certificate is OK, under circumstances when it is not. You might find of interest this official vendor declaration about the code when it was initially realized what was happening, and a quick fix was put in place: “Secure Transport failed to validate the authenticity of the connection. This issue was addressed by restoring missing validation steps.” Basically, you could potentially exploit the bug by tricking a device that was connecting to a web site and place yourself into the middle, doing so to surreptitiously watch and read the traffic going back-and-forth, grabbing up private info which you might use for nefarious purposes. This is commonly known as the Man-in-the-Middle security attack (MITM). I’ve now provided you with an example of a hole in the dam. It is a seemingly small hole, yet it undermined a much larger dam. Among a length chain of things that need to occur for the security aspects of the SSL/TSL, this one weak link undermined a lot of it. I do want to make sure that you know that it was not completely undermined since some parts of the code were working as intended and it was this particular slice that had the issue. There are an estimated 2,000 lines of code in this one program. Out of the 2,000 lines of code, one line, the infamous extra “goto fail” had caused the overall program to now falter in terms of what it was intended to achieve. That means that only 0.05% of the code was “wrong” and yet it undermined the entire program. Some would describe this as an exemplar of being brittle. Presumably, we don’t want most things in our lives to be brittle. We want them to be robust. We want them to be resilient. The placement of just one line of code in the wrong spot and then undermining a significant overall intent is seemingly not something we would agree to be properly robust or resilient. Fortunately, this instance did not seem to cause any known security breeches to get undertaken and no lives were lost. Imagine though that this were to happen inside a real-time system that is controlling a robotic arm in a manufacturing plant. Suppose the code worked most of the time, but on a rare occasion it reached a spot of this same kind of unconditional GO TO, and perhaps jumped past code that checks to make sure that a human is not in the way of the moving robotic arm. By bypassing that verification code, the consequences could be dire. For the story of the Dutch boy that plugged the hole in the dam, we are never told how the hole got there in the first place. It is a mystery, though most people that read the story just take it at face value that there was a hole. I’d like to take a moment and speculate about the infamous GO TO of the “goto fail” and see if we can learn any additional lessons by doing so, including possibly how it go there. Nobody seems to know how it actually happened, well, I’m sure someone does that was involved in the code (they aren’t saying). Anyway, let’s start with the theories that I think are most entertaining but seem farfetched, in my opinion. One theory is that it was purposely planted into the code, doing so at the request of someone such as perhaps the NSA. It’s a nifty theory because you can couple with it that the use of the single GO TO statement makes the matter seem as though it was an innocent mistake. What better way to plant a backdoor and yet if it is later discovered you can say that it was merely an accident all along. Sweet! Of course, the conspiracy theorists say that’s what they want us to think, namely that it was just a pure accident. Sorry, I’m not buying into the conspiracy theory on this. Yes, I realize it means that maybe I’ve been bamboozled. For conspiracy theories in the AI field, see my article: Another theory is that the programmer or programmers (we don’t know for sure if it was one programmer, and so maybe it was several that got together on this), opted to plant the GO TO statement and keep it in their back pocket. This is the kind of thing you might try to sell on the dark web. There are a slew of zero-day exploits that untoward hackers trade and sell, so why not do the same with this? Once again, this seems to almost make sense because the beauty is that the hole is based on just one GO TO statement. This might provide plausible deniability if the code is tracked to whomever put the GO TO statement in there. For my article about security backdoor holes, see: For my article about stealing of software code aspects, see: For aspects of reverse engineering code, see my article: I’m going to vote against this purposeful hacking theory. I realize that I might be falling for someone’s scam and they are laughing all the way to the bank about it. I don’t think so. In any case, now let’s dispense with those theories and got toward something that I think has a much higher chance of approaching what really did happen. ‘Mistakenly Done’ Theories First, we’ll divide the remaining options into something that was mistakenly done versus something intentionally done. I’ll cover the “mistakenly done” theories first. You are a harried programmer. You are churning out gobs of code. While writing those IF statements, you accidentally fat finger an extra “goto fail” into the code. At the time, you’ve indented it and so it appears to be in the right spot. By mistake, you have placed that line into your code. It becomes part of the landscape of the code. That’s one theory about the mistaken-basis angle. Another theory is that the programmer had intended to put another IF statement into that segment of the code and had typed the “goto fail” portion, but then somehow got distracted or interrupted and neglected to put the first part, the IF statement part itself, into the code. Yet another variation is that there was an IF statement there, but the programmer for some reason opted to delete it, but when the programmer did the delete, they mistakenly did not remove the “goto fail” which would have been easy to miss because it was on the next physical line. We can also play with the idea that there might have been multiple programmers involved. Suppose one programmer wrote part of that portion with the IF statements, and another programmer was also working on the code, using another instance, and when the two instances got merged together, the merging led to the extra GO TO statement. On a similar front, there is a bunch of IF statements earlier in the code. Maybe those IF statements were copied and used for this set of IF statements, and when the programmer or programmers were cleaning up the copied IF statements, they inadvertently added the unconditional GO TO statement. Let’s shift our attention to the “intentional” theories of how the line got in there. The programmer was writing the code and after having written those series of IF statements, took another look and thought they had forgotten to put a “goto fail” for the IF statement that precedes the now known to be wrong GO TO statement. In their mind, they thought they were putting in the line because it needed to go there. Or, maybe the programmer had been doing some testing of the code. While doing testing, the programmer opted to temporarily put the GO TO into the series of IF statements, wanting to momentarily short circuit the rest of the routine. This was handy at the time. Unfortunately, the programmer forgot to remove it later on. Or, another programmer was inspecting the code. Being rushed or distracted, the programmer thought that a GO TO opt to be in the mix of those IF statements. We know now that this isn’t a logical thing to do, but perhaps at the time, in the mind of the programmer, it was conceived that the GO TO was going to have some other positive effect, and so they put it into the code. Programmers are human beings. They make mistakes. They can have one thing in mind about the code, and yet the code might actually end-up doing something other than what they thought. Some people were quick to judge that the programmer must have been a rookie to have let this happen. I’m not so sure that we can make such a judgment. I’ve known and managed many programmers and software engineers that were topnotch, seasoned with many years of complex systems projects, and yet they too made mistakes, doing so and yet at first insistent to the extreme that they must be right, having recalcitrant chagrin afterward when proven to be wrong. This then takes us to another perspective, namely if any of those aforementioned theories about the mistaken action or the intentional action are true, how come it wasn’t caught? Typically, many software teams do code reviews. This might involve merely having another developer eyeball your code, or it might be more exhaustive and involve you walking them through it, including each trying to prove or disprove that the code is proper and complete. Would this error have been caught by a code review? Maybe yes, maybe not. This is somewhat insidious because it is only one line, and it was indented to fall into line with the other lines, helping to mask it or at least camouflage it by appearing to be nicely woven into the code. Suppose the code review was surface level and involved simply eyeballing the code. That kind of code review could easily miss catching this GO TO statement issue. Suppose it was noticed during code review, but it was put to the side for a future look-see, and then because the programmers were doing a thousand things at once, oops it got left in the code. That’s another real possibility. For my article about burned out developers, see: For the egocentric aspects of programmers, see my article: For my article about the dangers of groupthink and developers, see: You also need to consider the human aspects of trust and belief in the skills of the other programmers involved in a programming team. Suppose the programmer that wrote this code was considered topnotch. Time after time, their code was flawless. On this particular occasion, when it came to doing a code review, it was a slimmer code review because of the trust placed in that programmer. When managing software engineers, they sometimes will get huffy at me when I have them do code reviews. There are some that will say they are professionals and don’t need a code review, or that if there is a code review it should be quick and lite because of how good they are. I respect their skill sets but try to point out that any of us can have something mar our work. One aspect that is very hard to get across involves the notion of egoless coding and code reviews. The notion is that you try to separate the person from the code in terms of the aspect that any kind of critiquing of the code becomes an attack on that person. This means that no one wants to do these code reviews when it spirals downward into a hatred fest. What can happen is the code reviews become an unseemly quagmire of accusations and anger, spilling out based not only on the code but perhaps due to other personal animosity too. Besides code reviews, one could say that this GO TO statement should have been found during testing of the code. Certainly, it would seem at the unit level of testing, you could have setup a test suite of cases that fed into this routine, and you would have discovered that sometimes the verification was passing when it should not be. Perhaps the unit testing was done in a shallow way. We might also wonder what happened at doing a system test. Normally, you put together the various units or multiple pieces and do a test across the whole system or subsystem. If they did so, how did this get missed? Again, it could be that the test cases used at the system level did not encompass anything that ultimately rolled down into this particular routine and would have showcased the erroneous result. You might wonder how the compiler itself missed this aspect. Some compilers can do a kind of static analysis trying to find things that might be awry, such as dead code. Apparently, at the time, there was speculation that the compiler could have helped, but it had options that were either confusing to use, or when used were often mistaken in what they found. We can take a different perspective and question how the code itself is written and structured overall. One aspect that is often done but should be typically reconsidered is that the “err” value that gets used in this routine and sent back to the rest of the software was set initially to being Okay, and only once something found an untoward does it get set to a Not Okay signal. This meant that when the verification code was skipped, the flag was defaulting to everything being Okay. One might argue that this is the opposite of the right way to do things. Maybe you ought to assume that the verification is Not Okay, and the routine has to essentially go through all the verifications to set the value to Okay. In this manner, if somehow the routine short circuits early, at least the verification is stated as Not Okay. This would seem like a safer default in such a case. Another aspect would be the use of curly braces or brackets. Remember that I had earlier stated you can use those on an IF statement. Beside having use for multiple statements on an IF, it also can be a visual indicator for a human programmer of the start and end of the body of statements. Some believe that if the programmer had used the curly braces, the odds are that the extra “goto fail” would have stuck out more so as a sore thumb. We can also question the use of the multiple IF’s in a series. This is often done by programmers, and it is a kind of easy (some say sloppy or lazy) way to do things, but there are other programming techniques and constructs that can be used instead. Ongoing Debate on Dangers of GO TO Statements There are some that have attacked the use of the GO TO statements throughout the code passage. You might be aware that there has been an ongoing debate about the “dangers” of using GO TO statements. Some have said it is a construct that should be banned entirely. Perhaps the debate was most vividly started when Edgar Dijkstra had his letter published in the Communications of the ACM in March of 1968. The debate about the merits versus the downsides of the GO TO have continued since then. You could restructure this code to eliminate the GO TO statements, in which case, the extra GO TO would never have gotten into the mix, presumably. Another aspect involves the notion that the “goto fail” is repeated in the offending portion, which some would say should have actually made it visually standout. Would your eye tend to catch the same line of code repeated twice like this, especially a somewhat naked GO TO statement? Apparently, presumably, it did not. Some say the compiler should have issued a warning about a seemingly repeated line, even if it wasn’t set to detect dead code. You might also point out that this code doesn’t seem to have much built-in self-checking going on. You can write your code to “just get the job done” and it then provides its result. Another approach involves adding additional layers of code that do various double-checks. If that had been built into this code, maybe it would have detected that the verification was not being done to a full extent, and whatever error handling should take place would then have gotten invoked. In the software field, we often speak of the smell of a piece of code. Code-smell means that the code might be poorly written or suspect in one manner or another, and upon taking a sniff or a whiff of it (by looking at the code), one might detect a foul odor, possibly even a stench. Software developers also refer to technical debt. This means that when you right somewhat foul code, your creating a kind of debt that will someday be due. It’s like taking out a loan, and eventually the loan will need to be paid back. Bad code will almost always boomerang and eventually come back to haunt. I try to impart among my software developers that we ought to be creating technical credit, meaning that we’ve structured and written the code for future ease of maintenance and growth. We have planted the seed for this, even if at the time that we developed the code we didn’t necessarily need to do so. As a long-time programmer and software engineer, I am admittedly sympathetic to… Read more »
  • Despite Concerns, AI Making Inroads in Human Resources
    It’s no longer shocking that human resources departments use artificial intelligence. In fact, according to Littler’s 2018 Annual Employer Survey, 49 percent said they use AI and advanced data analytics for recruiting and hiring. They also deploy AI into HR-related activities such as: Making strategic and employee management decisions (31 percent). Analyzing workplace policies (24 percent). Automating certain tasks that were previously done by an employee (22 percent). So, where can HR leaders expect to see significant gains in how AI will support HR-driven uses cases? The experts weigh in. AI Risk of Bias in HR There are some caveats to consider with AI-infused human resources initiatives. For starters, companies should keep a close eye on how these AI tools perform as they risk inadvertently introducing bias, according to Armen Berjikly, head of AI at Ultimate Software. Last year at this time, researchers from MIT and Stanford University found that three commercially released facial-analysis programs from major technology companies demonstrated both skin-type and gender biases. “The most significant risk of AI-enabled recruiting is that AI doesn’t take risks,” Berjikly said. “An AI-enabled hiring process gets extremely good at finding the types of candidates you train it to find, which leaves out many potentially amazing applicants who don’t fit the proverbial mold.” Moving Forward with AI Despite Job-Loss Concerns And surely, there are the natural worries about AI being so efficient for departments like HR that it will eliminate jobs. Their worries are validated, robots are already conducting job interviews. Rohit Chawla, co-founder of Bridging Gaps, said he strongly feels AI will take the load off at least 25 to 30 percent of mundane HR jobs. While that may produce fear of humans losing jobs, it’s not time for companies to back away from time-saving AI initiatives for HR. “HR should [embrace the technology] as currently a lot of customer-facing aspects are being taken care by AI. It’s high time HR takes up the challenge without any fear,” he added. Chawla, who raised questions of using AI in HR scenarios, sees these common areas where AI is helping human resources: Searching right-fit candidates especially for junior-level positions. Similarly conducting AI-based interviews both behavioral and functional. Sharing regret information to rejected candidates with an extent of sharing the reason, also not possible manually. Using chatbots to resolve employee queries. Workforce Data Leads to Predictive Advantage Where else is AI winning in HR? Jayson Saba, senior director of product marketing at Kronos, said AI advancements in HR are helping organizations leverage transactional workforce data to predict employee potential, fatigue, flight risk and even overall engagement. This enables more productive conversations to improve the employee experience, retention and performance. “It’s now possible to leverage AI to build smarter, personalized schedules and to leverage AI to review time-off and shift-swap requests in real-time based on predetermined business rules,” he said. This empowers employees, especially those with front-line/hourly positions, to take more control of their work/life balance. “Using AI for these important but repetitive administrative requests also unburdens managers, allowing them to spend more time on the floor, working with customers and training teams,” Saba added. Intelligent Shift-Swapping Real-time analytics can show managers the impact that absences, open shifts and unplanned schedule changes will have on key performance indicators, allowing them to make more informed decisions that avoid issues before they arise. Similarly, Saba said, using an intelligent solution to automate shift-swapping without manager intervention reduces the number of last-minute call-outs, no-shows and vacant shifts and effectively removes the need to schedule additional labor to cover for anticipated absences. “The future of work in any industry is going to rely heavily on advances in AI for HR,” Saba added, “but it’s important to keep in mind that AI will never replace the manager. Instead, its true value is analyzing the massive amounts of workforce data to provide managers with better informed options to guide their decisions.” Read the source article in Read more »
  • How AI and Machine Learning Are Improving Manufacturing Productivity
    Engineers at the Advanced Manufacturing Research Centre’s Factory 2050 in Sheffield, UK are using Artificial Intelligence (AI) to learn what machine utilization looks like on the workshop floor. The aim is to create a demonstrator to show just how accessible Industry 4.0 technologies are, and how they can potentially revolutionize shop-floor productivity. The demonstrator will be the first created under an emerging AI strategy being produced at Factory 2050, which seeks to harness the innovative work being done with AI and machine learning techniques across the Advanced Manufacturing Research Centre (AMRC) and provide real use-cases for these techniques in industrial environments. “Using edge computing devices retrofitted to CNC machines, we have collected power consumption data during the production of automotive suspension components,” said Rikki Coles, AI Project Engineer for the AMRC’s Integrated Manufacturing Group at Factory 2050. “It isn’t a complicated parameter to measure on a CNC machine, but using AI and machine learning, we can actually do a lot with such simple data.” Data from the edge computing devices at from partner Tinsley Bridge was sent to the AMRC’s cloud computing services, and using the latest data science techniques, ran through an AI algorithm to provide new insights for the control and monitoring of manufacturing processes. Analyzing the power signatures from the data, the algorithm looked for repeating patterns or anomalies, working out how many components were machined and deduced that three different types of components were manufactured. Rikki said: “The project demonstrates to industry that with a low cost device collating quite simple data, AI and machine learning can be used to create valuable insights from this data for the manufacturer.” Director of Engineering at Tinsley Bridge, Russell Crow, said: “Interrogating our machine utilization rates means we have better visibility of what was being manufactured and when, and the ability to assess if we are scheduling effectively.  This data will allow us to look at boosting our productivity on the shop floor.” “Rather than investing in significant cost and time for new digitally integrated smart machining centres, we were able to work with the AMRC to retrofit our existing capabilities to achieve the same results and enhance what data we were collecting by fitting a simple current clamp to our machines; an unobtrusive solution that caused no disruption or downtime.” Aiden Lockwood, Chief Enterprise Architect for the AMRC Group said the project demonstrator will show other SMEs how easily and cheaply Industry 4.0 technologies can be accessed: “Traditionally these tools were built into commercial packages which could be out of the reach for some SMEs, so there is a misunderstanding that Industry 4.0 manufacturing techniques are for the big players who handle incredibly complex data collected over a long period of time.” “But AI is evolving and these techniques now give smaller businesses the ability to do so much more with their data. In this project we are using a simple data set, collected over a short period of time to provide real benefits for the company.” Aiden said the AMRC want to show what using AI in manufacturing looks like for small businesses: “The formation of our AI strategy will allow us to lead the way in developing new capabilities and bringing the academic, tech and business communities of the region together to educate and demonstrate AI technologies for manufacturing industries; learning from developments in retail, finance and marketing.” The next phase of the project will see the engineers at the AMRC train the system further so the algorithm can detect non-conforming components whilst in production, or identify a problem when a machine is requiring intervention, such as inconsistent tool wear which affects component quality. “Alongside the power consumption data, the plan is to feed the algorithm with available data about which of the manufactured components were non-conforming. So as well as providing clarity around machine utilization, the algorithm will essentially learn what a ‘good’ manufacturing process looks like and be able to actively monitor on-going manufacturing processes,” said Rikki. Read the source article in Read more »
  • How AI Can Help Solve Some of Humanity’s Greatest Challenges
    By Marshall Lincoln and Keyur Patel, cofounders of the Lucid Analytics Project In 2015, all 193 member countries of the United Nations ratified the 2030 “Sustainable Development Goals” (SDG): a call to action to “end poverty, protect the planet and ensure that all people enjoy peace and prosperity.” The 17 goals – shown in the chart below – are measured against 169 targets, set on a purposefully aggressive timeline. The first of these targets, for example, is: “by 2030, [to] eradicate extreme poverty for all people everywhere, currently measured as people living on less than $1.25 a day”. The UN emphasizes that Science, Technology and Innovation (STI) will be critical in the pursuit of these ambitious targets. Rapid advances in technologies which have only really emerged in the past decade – such as the internet of things (IoT), blockchain, and advanced network connectivity – have exciting SDG applications. No innovation is expected to be more pervasive and transformative, however, than artificial intelligence (AI) and machine learning (ML). A recent study by the McKinsey Global Institute found that AI could add around 16 per cent to global output by 2030 – or about $13 trillion. McKinsey calculates that the annual increase in productivity growth it engenders could substantially surpass the impact of earlier technologies that have fundamentally transformed our world – including the steam engine, computers, and broadband internet. AI/ML is not only revolutionary in its own right, but also increasingly central to the foundation upon which the next generation of technologies are being built. But the pace and scale of the change it will bring about also creates risks that humanity must take very seriously. Our research has led us to conclude that AI/ML will directly contribute to at least 12 of the 17 SDGs – likely more than any other emerging technology. In this piece, we explore potential use cases in three areas which are central to the Global Goals: financial inclusion, healthcare and disaster relief, and transportation. FINANCIAL INCLUSION Access to basic financial services – including tools to store savings, make and receive payments, and obtain credit and insurance – are often a prerequisite to alleviating poverty. Around 2 billion people around the world have limited or no access to these services. AI/ML is increasingly helping financial institutions create business models to serve the unbanked. For example, one of the biggest barriers to issuing loans is that many individuals and micro businesses have no formal credit history. Start-ups are increasingly running ML algorithms on non-traditional sources of data to establish their creditworthiness – from shopkeepers’ orders and payments history to psychometric testing. Analysis of data on crop yields and climate patterns can be used to help farmers use their land more effectively – reducing risks for lenders and insurance providers. AI/ML is also being used to help service providers keep their costs down in markets where revenue per customer is often very small. These include automated personal finance management, customer service chat-bots, and fraud detection mechanisms. HEALTHCARE AND DISASTER RELIEF The inequality between urban and rural healthcare services is an urgent problem in many developing countries. Rural areas with poor infrastructure often suffer from severe shortages of qualified medical professionals and facilities. Smart phones and portable health devices with biometric sensors bring the tools of a doctor’s office to patients’ homes – or a communal location in a village center for shared use. AI then automates much of the diagnostic and prescriptive work traditionally performed by doctors. This can reduce costs, enable faster and more accurate diagnoses, and ease the burden on overworked healthcare workers. AI is also being used to get medical supplies where they are needed. A start-up called Zipline, for example, is using AI to schedule and coordinate drones to deliver blood and equipment to rural areas in Rwanda (and soon other countries in Africa) which are difficult to access by road. Doctors order what they need via a text messaging system, and AI handles delivery. This dramatically reduces the time it takes to obtain blood in an emergency and eliminates wastage. When it comes to disaster relief, predictive models – based on data from news sources, social media, etc. – can help streamline crisis operations and humanitarian assistance. For example, AI-powered real-time predictions about where earthquakes or floods will cause the most damage can help emergency crews decide where to focus their efforts. Read the source article at KDNuggets.   Read more »
  • Machine Learning Engineer vs. Data Scientist—Who Does What?
    The roles of machine learning engineer vs. data scientist are both relatively new and can seem to blur. However, if you parse things out and examine the semantics, the distinctions become clear. At a high level, we’re talking about scientists and engineers. While a scientist needs to fully understand the, well, science behind their work, an engineer is tasked with building something. But before we go any further, let’s address the difference between machine learning and data science. It starts with having a solid definition of artificial intelligence. This term was first coined by John McCarthy in 1956 to discuss and develop the concept of “thinking machines,” which included the following: Automata theory Complex information processing Cybernetics Approximately six decades later, artificial intelligence is now perceived to be a sub-field of computer science where computer systems are developed to perform tasks that would typically demand human intervention. These include: Decision-making Speech recognition Translation between languages Visual perception Machine learning is a branch of artificial intelligence where a class of data-driven algorithms enables software applications to become highly accurate in predicting outcomes without any need for explicit programming. The basic premise here is to develop algorithms that can receive input data and leverage statistical models to predict an output while updating outputs as new data becomes available. The processes involved have a lot in common with predictive modeling and data mining. This is because both approaches demand one to search through the data to identify patterns and adjust the program accordingly. Most of us have experienced machine learning in action in one form or another. If you have shopped on Amazon or watched something on Netflix, those personalized (product or movie) recommendations are machine learning in action. Data science can be described as the description, prediction, and causal inference from both structured and unstructured data. This discipline helps individuals and enterprises make better business decisions. It’s also a study of where data originates, what it represents, and how it could be transformed into a valuable resource. To achieve the latter, a massive amount of data has to be mined to identify patterns to help businesses: Gain a competitive advantage Identify new market opportunities Increase efficiencies Rein in costs The field of data science employs computer science disciplines like mathematics and statistics and incorporates techniques like data mining, cluster analysis, visualization, and—yes—machine learning. Having said all of that, this post aims to answer the following questions: Machine learning engineer vs. data scientist: what degree do they need? Machine learning engineer vs. data scientist: what do they actually do? Machine learning engineer vs. data scientist: what’s the average salary? Machine Learning Engineer vs. Data Scientist: What They Do As mentioned above, there are some similarities when it comes to the roles of machine learning engineers and data scientists. However, if you look at the two roles as members of the same team, a data scientist does the statistical analysis required to determine which machine learning approach to use, then they model the algorithm and prototype it for testing. At that point, a machine learning engineer takes the prototyped model and makes it work in a production environment at scale. Going back to the scientist vs. engineer split, a machine learning engineer isn’t necessarily expected to understand the predictive models and their underlying mathematics the way a data scientist is. A machine learning engineer is, however, expected to master the software tools that make these models usable. What Does a Machine Learning Engineer Do? Machine learning engineers sit at the intersection of software engineering and data science. They leverage big data tools and programming frameworks to ensure that the raw data gathered from data pipelines are redefined as data science models that are ready to scale as needed. Machine learning engineers feed data into models defined by data scientists. They’re also responsible for taking theoretical data science models and helping scale them out to production-level models that can handle terabytes of real-time data. Machine learning engineers also build programs that control computers and robots. The algorithms developed by machine learning engineers enable a machine to identify patterns in its own programming data and teach itself to understand commands and even think for itself. What Does a Data Scientist Do? When a business needs to answer a question or solve a problem, they turn to a data scientist to gather, process, and derive valuable insights from the data. Whenever data scientists are hired by an organization, they will explore all aspects of the business and develop programs using programming languages like Java to perform robust analytics. They will also use online experiments along with other methods to help businesses achieve sustainable growth. Additionally, they can develop personalized data products to help companies better understand themselves and their customers to make better business decisions. As previously mentioned, data scientists focus on the statistical analysis and research needed to determine which machine learning approach to use, then they model the algorithm and prototype it for testing. What Do the Experts Say? Springboard recently asked two working professionals for their definitions of machine learning engineer vs. data scientist. Mansha Mahtani, a data scientist at Instagram, said: “Given both professions are relatively new, there tends to be a little bit of fluidity on how you define what a machine learning engineer is and what a data scientist is. My experience has been that machine learning engineers tend to write production-level code. For example, if you were a machine learning engineer creating a product to give recommendations to the user, you’d be actually writing live code that would eventually reach your user. The data scientist would be probably part of that process—maybe helping the machine learning engineer determine what are the features that go into that model—but usually data scientists tend to be a little bit more ad hoc to drive a business decision as opposed to writing production-level code.” Shubhankar Jain, a machine learning engineer at SurveyMonkey, said: “A data scientist today would primarily be responsible for translating this business problem of, for example, we want to figure out what product we should sell next to our customers if they’ve already bought a product from us. And translating that business problem into more of a technical model and being able to then output a model that can take in a certain set of attributes about a customer and then spit out some sort of result. An ML engineer would probably then take that model that this data scientist developed and integrate it in with the rest of the company’s platform—and that could involve building, say, an API around this model so that it can be served and consumed, and then being able to maintain the integrity and quality of this model so that it continues to serve really accurate predictions.” Read the source post on the Springboard Blog.  Read more »
  • Boxes-on-Wheels and AI Self-Driving Cars
    By Lance Eliot, the AI Trends Insider Watch out, the rolling boxes are on their way. Many call them a box-on-wheels. That’s referring to the use of AI self-driving car technology to have a vehicle that would be driverless and would deliver goods to you or more. At the Cybernetic AI Self-Driving Car Lab, we are developing AI software for self-driving cars, and are also including into our scope the use of AI systems for boxes-on-wheels. I offer next some salient aspects about the emerging niche of boxes-on-wheels. For my framework on AI self-driving cars, see: Some falsely believe autonomous shuttles are the same as box-on-wheels designs, not so, see my article: For the grand convergence that’s leading to these AI autonomously driven vehicles, see my article: Let’s start with a typical use case for a box-on-wheels. You could potentially order your groceries online from your neighborhood grocer, and a little while later those groceries pull-up in front of your house as contained in a so-called box-on-wheels. You walk outside to the vehicle, enter a special PIN code or some other form of recognition, and inside are your groceries. You happily carry the grocery bags up to your apartment or house and do so without ever having to drive your car. The vehicle drives off to deliver groceries to others that have also made recent orders from that grocery store. Notice that I mentioned that this is considered a use of AI self-driving car technology. It is not the same as what most people think of as an AI self-driving car per se. I say that because the vehicle itself does not necessarily need to look like a passenger car. A box-on-wheels can be a different shape and size than a normal passenger car, since it is not intending to carry humans.  It is intended to carry goods. If you ponder this aspect of carrying goods, you’d likely realize that it would be best to design the vehicle in a manner intended to carry goods rather than carrying humans. Consider first what it’s like to carry goods inside a passenger car. I’m sure you’ve tried to pile your own grocery bags into the backseat of your car or maybe on the floor just ahead of the passenger front seat. The odds are that at some point you had those bags flop over and spill their contents. If you made a quick stop by hitting the brakes of the car, it could be that you’ve had groceries that littered throughout your car and maybe had broken glass from a smashed milk bottle as a result. Not good. Don’t blame it on the passenger car! The passenger car is considered optimized to carry people. There are seats for people. There are armrests for people. There are areas for people to put their feet. All in all, the typical passenger car is not particularly suited to carry goods. Sure, you might place the goods into your trunk or maybe some other baggage carrying spaces of the car, but then you’d be unable to use the passenger seats in any sensible way to carry goods. Nope, don’t try to make a hammer into a screwdriver. If you need a hammer, get yourself a hammer. If you need a screwdriver, get yourself a screwdriver. Thus, I think you can understand the great value and importance of developing a vehicle optimized for carrying goods, of which it is not bound to the design of a passenger carrying car. There are a wide variety of these designs all vying to see which will be the best, or at least become more enduring, as to meeting the needs of delivering goods. Some of these vehicles are the same size as a passenger car. Some of these vehicles are much smaller than a passenger car, of which, some of those are envisioned to go on sidewalks rather than solely on the streets. The ones that go on the sidewalks need to especially be honed to cope with pedestrians and other aspects of driving on a sidewalk, plus there often is the need to get regulatory approval in a particular area to allow a motorized vehicle to go on sidewalks. Having such a vehicle on a sidewalk can be a dicey proposition. If you are wondering why even try, the notion is that it can more readily get to harder to reach places due to its smaller size and overall footprint, and in neighborhoods where they restrict the use of full sized cars it could potentially do the delivery (such as retirement communities), even perhaps right up to the door of someone’s adobe. Some designers are going to the opposite extreme and considering boxes-on-wheels that are the size of a limo or larger. The logic is that you could store even more groceries or other goods in one that is larger in size. This could cut down on the number of trips needed to deliver some N number of goods to Y number of delivery spots. Suppose a “conventional” box-on-wheels allowed for up to 6 distinct deliveries, while the limo version could do say twelve. The box-on-wheels for the six distinct deliveries would need to come all the way back to the grocery store to fill-up the next set of six, meanwhile the limo version would have gotten all twelve put into it at the start of its journey and would be more efficient to deliver them without having to come back mid-way of the twelve. The downside of the limo sized box-on-wheels is whether it can readily navigate the roads needed to do its delivery journey. With a larger size, it might not be able to make some tight corners or other narrow passages to reach the intended recipient of the goods. There’s a trade-off between the size of the box-on-wheels and where it can potentially go. Indeed, let’s be clear that there is no one-size-fits-all solution here. There are arguments about which of the sizes will win out in the end of this evolving tryout of varying sizes and shapes of boxes-on-wheels. I am doubtful there will be only one “right size and shape” that will accommodate the myriad of needs for a boxes-on-wheels. Just as today we have varying sizes of cars and trucks, the same is likely to be true for the boxes-on-wheels. For my article about safety aspects of AI self-driving vehicles, see: For various AI self-driving vehicle design aspects, see my article: For the myth of these vehicles becoming economic commodities, see my myth busting article: Box on Wheels Free-for-All Today That doesn’t though suggest that all of the variants being tried today will survive. I’m sure that many of the designs of today will either morph and be revised based on what seems to function well in the real-world, or some designs will be dropped entirely, or other new designs will emerge once we see what seems to work and what does not work. It’s a free-for-all right now. Large sized, mid-sized, small-sized, along with doors that open upward, downward, or swing to the side, and some with windows and others without windows, etc. Let’s consider an example of a variant being tried out today. Kroger, a major grocer, has teamed up with Nuro, an AI self-driving vehicle company, for the development of and testing of delivery vehicles that would carry groceries. The squat looking vehicle has various separated compartments to put groceries into. There are special doors that can be opened to then allow humans to access the compartments, presumably for the purposes of putting in groceries at the grocery store and then taking out the groceries when the vehicle reaches the consumer that bought the groceries. This kind of design makes a lot of sense for the stated purpose of transporting groceries. You want to have separated compartments so that you could accommodate multiple separate orders. Maybe you ordered some groceries, and Sam that lives two blocks away also ordered groceries. Naturally, you’d not want Sam to mess around with your groceries, and likewise you shouldn’t mess around with Sam’s groceries. Imagine if you could indeed access other people’s groceries – it could be a nightmare of accidentally taking the wrong items (intended for someone else), or accidentally crushing someone else’s items (oops, flattened that loaf of bread), and maybe intentionally doing so (you’ve never liked Sam, so you make sure all the eggs he ordered are smashed). There has to be also be some relatively easy way to access the compartments. Having a lockable door would be essential. The door has to swing or hinge in a manner that it would be simple to deal with and allow you access to the compartment readily and fully. You of course don’t want humans to get confused trying to open or close the doors. You don’t want humans to hurt themselves when opening or closing a door. The locking mechanism has to allow for an easy means of identifying the person that is rightfully going to open the door. And so on. The locking mechanism might involve you entering a PIN code to open the door. The PIN would have been perhaps provided to you when you placed your grocery order. Or, it might be that your smartphone can activate and unlock the compartment door, using NFC or other kinds of ways to convey a special code to the box-on-wheels. It could even be facial recognition or via your eye or fingerprint recognition, though this means that only you can open the door. I say this because you might be unable to physically get to the box-on-wheels and instead have someone else aiding you, maybe you are bedridden with some ailment and have an aid in your home, and so if the lock only responds to you it would limit your allowing someone else to open it instead (possibly, you could instruct the lock via online means as to how you want it to respond). I mention these aspects because the conventional notion is that the box-on-wheels will most likely be unattended by a human. If you had a human attendant that was inside the vehicle, they could presumably get out of the vehicle when it reaches your home, they could open the door to the compartment that contains your groceries, and they might either hand it to you or walk it up to your door. But, if the vehicle is unattended by a human, this means that the everyday person receiving the delivery is going to have to figure out how to open the compartment door, take out the groceries, and then close the compartment door. This seems like a simple task, but do not underestimate the ability of humans to get confused at tasks that might seem simple on the surface, and also be sympathetic towards those that might have more limited physical capabilities and cannot readily perform those physical tasks. Presumably, the compartment doors will have an automated way to open and close, rather than you needing to physically push open and push closed the compartment doors (though, not all designs are using an automated door open/close scheme). This does bring up some facets about these boxes on wheels that you need to consider. First, there’s the aspect of having a human on-board versus not having a human on-board:         Human attendant         No human attendant I’ve carefully phrased this to say human attendant. We don’t need to have a human driver in these vehicles since the AI is supposed to be doing the driving. This though does not imply that the vehicle has to be empty of a human being in it. You might want to have a human attendant in the vehicle. The human attendant would not need to know how to drive. Indeed, even if they knew how to drive, the vehicle would most likely have no provision for a human to drive it (there’d not be any pedals or steering wheel). Why have a human attendant, you might ask? Aren’t we trying to take the human out of the equation by using the AI self-driving car technology? Well, you might want to have a human attendant for the purposes of attending to the vehicle when needed. For example, suppose the grocery carrying vehicle comes up to my house and parks at the curb in front of my house. Darned if I broke my leg in a skiing incident a few weeks ago and I cannot make my way out to the curb. Even if I could hobble to the curb, I certainly couldn’t carry the grocery bags back into the house and hobble at the same time. The friendly attendant instead leaps out of the vehicle when it reaches my curb. They come up to my door, ring the doorbell, and provide me with my grocery bags. I’m so happy that I got my groceries brought to my door and did not have to hassle going out to the vehicle. This could be true too if you were in your pajamas or maybe drunken from that wild party taking place in your home. The “last mile” of having a vehicle pull-up to your curb, or perhaps park in your driveway, or wherever, the AI self-driving car system isn’t going to bridge that gap. Having a human attendant would. Think too that the human attendant does not need to know how to drive a car and doesn’t need a driver’s license. Therefore, the skill set of the human attendant is quite a bit less than if you had to hire a driver. Also, the AI is doing the driving and so you don’t need to worry about whether the human attendant got enough sleep last night to properly drive the box-on-wheels. Essentially, this human attendant is the equivalent of the “box boy” (or “box girl”) that boxes up your groceries in the store (well, that’s in stores that still do so). Having a human attendant can be a handy “customer service” aspect. They can aid those getting a delivery, they can serve to showcase the humanness of the grocer, they can answer potential questions that the human recipient might have about the delivery, and so on. The downside is that by including the human attendant, you are adding cost to the delivery process, and you’ll also need to deal with the whole aspect of hiring (and firing) of the attendants. It could make deliveries a positive thing to have a human attendant, but it can also be a negative. If the human attendant is surly to the person receiving the goods, the humanness of things could backfire on the grocery store. Some say that the box-on-wheels should have a provision to include a human attendant, but then it would be up to the grocer to decide when to use human attendants or not. In other words, if the vehicle has no provision for a human attendant to ride on-board, the grocer then has no viable option to have the human attendant go along on the delivery. If you have the provision, you can then decide whether to deploy the human attendant or not, perhaps offering during certain hours of the day the human attendant goes along and at other times does not. Or, maybe that for an added fee, your grocery delivery will include an attendant and otherwise not. So, why not go ahead and include a space in the box-on-wheels to accommodate a human attendant? We’re back to the question of how to best design the vehicle. If you need to include an area of the vehicle that accommodates a human attendant, you then are sacrificing some of the space that could otherwise be used for the storing of the groceries. You also need to consider what must the requirements of this space consist of. For example, should it be at the front of the vehicle, akin to if the human was in the driver’s seat, or can it be in the back of someplace else. You would likely need to have a window for the person to see out of. There are various environmental conditions that the vehicle design would need to incorporate for the needs of a human. For future job roles as a result of the advent of AI self-driving vehicles, see my article: For my article on how Gen Z is going to shape the timeline of the advent of AI self-driving vehicles, see: For the potential of pranking AI self-driving vehicles, see my article: For my article about the public shaming of AI self-driving vehicles, see: This brings up another aspect about the box-on-wheels design, namely whether it can potentially do driving in a manner that would be beyond what a human would normally do. Assuming that the groceries are well secured and packaged into the compartments, the box-on-wheels could make sharp turns and brake suddenly, if it wanted or needed to do so. If there’s a human attendant on-board, those kinds of rapid maneuvers could harm the human, including perhaps some kind of whiplash or other injuries. Also, if the box-on-wheels somehow crashes or gets into an accident, if you have a human attendant on-board there needs to be protective mechanisms for them such as air bags and seat belts, while otherwise the only danger is to the groceries. I think we’d all agree that some bumped or smashed groceries is not of much concern, while a human attendant getting injured or maybe killed is a serious matter. Thus, another reason to not have a human attendant involves the risks of injury or death to the human, which if you are simply doing grocery delivery is adding a lot of risk to the attendant and to the grocer. Let’s shift attention now to the nature of the compartments that will be housing the goods. Grocery Bags in Compartmetns of the Box-on-Wheels For the delivery of groceries, it is so far assumed that the groceries will be placed into grocery bags and that in turn those grocery bags will be placed into the compartment of the box-on-wheels. This convention of our using grocery bags goes back many years (some say that the Deubner Shopping Bag invented in 1912 was the first modernized version) and seems to be a suitable way to allow humans to cart around their groceries (rather than perhaps cardboard boxes or other such containers). The grocery bags are quite handy in that they are something we all accept as a means of grouping together our groceries. It has a familiar look to it. Assuming that the grocery bag has some kind of straps, the manner in which you carry the grocery bag allows you to either carry it by the straps or you can carry the whole bag by picking it up from the bottom or grasping the bag in a bear hug.  In that sense, the grocery bag is a simple way allowing for multiple options as to how to carry it. This is mainly important for purposes of the human recipient and how they are to remove their groceries and then transport them into their adobe. For the moment, assume that the grocery store will indeed use a grocery bag for these purposes. You would want the grocery bag to be sturdy and not readily tear or fall apart – imagine if the box-on-wheels has no human attendant, arrives at the destination, and the human recipient pulls out their bag of groceries and it rips apart and all of their tangerines and other goods spill to the ground. The human recipient will be irked and likely not to order from that grocer again. Therefore, the odds are that the grocery bag being used for this purpose has to be as sturdy if not even more sturdy than getting a simple plastic bag or brown bag at your local grocery store. The odds are that the grocery store will use some kind of special cloth bag or equivalent which is durable and can safely hold the groceries and be transported. Likely the grocery store would brand the bags so that it is apparent they came from the XYZ grocery store. The twist to all of this is the cost of those bags and also what happens to them. The cost is likely high enough that it adds to the cost of the delivery overall. Also, if every time you receive a delivery you get and presumably keep the bags, it means that the grocer is going to be handing out a lot of these bags over time. Suppose I get about four bags of groceries every week, and I keep the bags, thus by the end of a year I’ve accumulated around 200 of these grocery bags! That’s a lot of grocery bags. You might say that the human recipient should put the grocery bags back into the box-on-wheels after emptying the grocery bags of their goods. That’s a keen idea. But, you probably don’t want the box-on-wheels to be sitting at the curb while the human recipient goes into their home, takes the groceries about of the bags, and then comes out to the box-on-wheels to place the empty grocery bags into it. This would be a huge delay to the box-on-wheels moving onward to deliver goods to the next person. So, this notion of the empty bag return would more likely need to be done when the human recipient gets their groceries, in that perhaps they might have leftover empty bags from a prior delivery and place those into the compartment when they remove their latest set of groceries. Then, when the box-on-wheels gets back to the grocery store, a clerk there would take out the empty grocery bags and perhaps credit the person with having returned them. This shifts our attention then to another important facet of the box-on-wheels, namely the use of the compartments. We’ve concentrated so far herein on the approach of delivering goods to someone. That’s a one-way view of things. The one-way that we’ve assumed in this discussion is that the grocery store is delivering something to the person that ordered the groceries. The human recipient removes their groceries from the compartment and the compartment then remains empty the rest of the journey of the box-on-wheels for the deliveries it is making in this round. Suppose though that the compartments were to be used for taking something from the person that received delivery goods. Or, maybe the compartment never had anything in it at all and arrived at the person’s home to pick-up something. The pick-up might be intended to then be delivered to the grocery store. Or, it could be that the pick-up is then delivered to someone else, like say Sam. As mentioned earlier, Sam lives some blocks away from you, and perhaps you have no easy means to send over something to him, and thus you use the grocery store box-on-wheels to do so. The possibilities seem endless. They also raise concerns. Do you really want people to put things into the compartments of the box-on-wheels? Suppose someone puts into a compartment a super stinky pair of old shoes, and it is so pungent that it mars the rest of the groceries in the other compartments? Or, suppose someone puts a can of paint in the compartment, fails to secure the lid of the paint can, and while the box-on-wheels continues its journey the paint spills all over the inside of the compartment. As you can see, allowing the recipient to put something into the compartment will be fraught with issues. Some grocers are indicating that the recipients will not be allowed to put anything into the compartments. This is perhaps the safest rule, but it also opens the question of how to enforce it. A person might put something into a compartment anyway. They might try to trick the system into carrying something for them. Ways to try and prevent this include the use of sensors in the compartment to try and detect whether anything is in the compartment, such as by weight or by movement. This does bring up an even more serious concern. There are some that are worried that these human unattended box-on-wheels could become a kind of joy ride for some. Imagine a teenager that “for fun” climbs into the compartment to go along for a ride. Or, maybe a jokester puts a dog into a compartment. Or, worse still, suppose someone puts their baby down into the compartment to lift out the grocery bag, and somehow forgets that they left their baby in the compartment (I know this seems inconceivable, but keep in mind there are a number of hot-car baby deaths each year, which illustrates that people can do these kinds of horrifying absent minded things). Besides having sensors in the compartments, another possibility involves the use of cameras on the box-on-wheels. There could be a camera inside each of the compartments, thus allowing for visual inspection of the compartment by someone remotely monitoring the box-on-wheels. You can think of this like the cameras these days that are in state-of-the-art refrigerators. Those cameras point inward into the refrigerator and you can while at work via your smartphone see what’s in your refrigerator (time to buy some groceries when the only thing left is a few cans of beer!). We can enlarge the idea of using cameras and utilize the cameras on the box-of-wheels that are there for the AI self-driving car aspects. Thus, once the box-on-wheels comes to a stop at the curb, it might be handy to still watch and see what happens after stopping. Presumably, you could see that someone is trying to put a dog into a compartment. The box-on-wheels might be outfitted with speakers and a remote operator could tell the person to not put a dog into the compartment. The use of remote operators raises added issues to the whole concept of the delivery of the goods. You are now adding labor into the process. How many remote operators do you need? Will you allow them to actually operate the box-on-wheels, or are they solely for purposes of acting like a human attendant? There are costs involved and other facets that make this a somewhat less desirable addition to the process. On the topic of remote operators, here’s another twist for you. Suppose the box-on-wheels arrives at the destination address. Turns out that the curb is painted red and presumably the box-on-wheels cannot legally stop there. The street is jam packed with parked cars. There is no place to come to a legal stop. What should the AI of the box-on-wheels do? We all know that a human driver would likely park temporarily at the red curb or might double-park the delivery vehicle. But, do we want the AI to act in an illegal manner? How else though will it solve the problem? You might say it needs to find a legal place to park, but that might be blocks away. You might say that people receiving the delivery will need to arrange for a legal place for the box-on-wheels to stop, but that’s a pretty tall order in terms of having to change the infrastructure of the street parking and dealing with local parking regulations, etc. For my article about the illegal driving aspects of AI self-driving cars, see: For the parking of AI self-driving cars, see my article: Some believe that with a remote human operator you might be able to deal with this parking issue by having the remote operator decide what to do. The remote operator, using the cameras of the AI self-driving vehicle, might be able to see and discern where to park the box-on-wheels. Would the remote operator directly control the vehicle? Some say yes, but if that’s the case then the question arises as to whether they need to be licensed to drive and opens another can of worms. Some therefore would say no, and that all the remote operator can do is make suggestions to the AI of where to park (“move over to that space two cars ahead”). This though can be a kind of splitting of hairs, since it might be interpreted that a remote operator giving parking instructions is no different than themselves actually driving the vehicle. For my article about remote operators of AI self-driving cars, see: Here’s another facet to consider. How long will the box-on-wheels be at a stopped position and allow for the removal of the goods? From the grocer viewpoint, you would want the stopped time to be the shortest possible. For every minute that the box-on-wheels sits at the curb and is waiting for the delivery to be completed, it is using up time to get to the next destination. Those further along in the delivery cycle are all waiting eagerly (or anxiously) for the box-on-wheels to get to them. Suppose a person comes out to the box-on-wheels, opens the compartment designated for their delivery, and for whatever reason rummages around in the grocery bag, maybe doing an inspection to make sure the bag contains what they ordered. They decide to then slowly remove the bag and slowly walk up to their home and slowly put the bag inside the home. Meanwhile, they have four other bags yet to go that are sitting in the compartment. They walk out slowly to get the next bag. And so on. If the system had calculated beforehand that it should take about four minutes to remove the bags by the recipient, it could be that this particular stop takes 20 minutes or even longer. How can you hurry along the recipient? If you had a human attendant, you’d presumably have a better chance of making the deliveries occur on a timelier basis. Without the human attendant, you could possibly use a remote human operator to urge someone to finish… Read more »
  • Thought Leadership: Tom Hebner of Nuance Communications
    Nuance Leveraging AI, Offers New Project Pathfinder Tool for Crafting Intelligent Conversation Apps Tom Hebner, Head of Product Innovation, Voice Technology and AI at Nuance Communications runs the team focused on creating new products and solutions for Voice Technology and Conversational AI. He recently took a few minutes to speak with AI Trends Editor John P. Desmond. Q. How is Nuance doing AI today? I know you have a long history in voice recognition. What makes what you do there now true AI? A. Good question, especially since we’re in an interesting world right now where it’s the perception of AI versus the reality of AI. What many people might not know is that speech recognition is AI technology. Natural language understanding is AI technology. Machine learning is applied AI technology. And just because it was around before the “AI revolution” doesn’t mean that it’s not AI. So, there’s a lot of buzz and hype right now. Tom Hebner of Nuance Along with that comes a lot of startups and even some of our friends in the big technology arena that are coming out and talking about conversational AI and AI in general, saying that there are these new capabilities and new things happening. We have speech recognition, as well as natural language processing. In fact, we have technologies that have been around for 20 years–technologies that have been tuned and optimized over the past two decades. Essentially, we have a very mature process around how to bring this AI technology to market. There is a perception that AI should be a single brain that knows and learns. However, the technology is not there yet. No one has delivered a single brain that totally understands somebody’s enterprise and converses with that organization’s customers because, frankly, that’s science fiction. It doesn’t exist yet. This all said, it doesn’t mean there isn’t strong AI-based technology that can be used to deliver solutions. Q. We do have a lot of buzz around AI these days, as you mentioned. What’s your view on where the industry needs to go? And what is Nuance doing to make it happen? A. Along with the buzz came a lot of “easy-to-use tools,” and claims like: “You can build a bot in a minute. You can build a bot in the afternoon. We’ll make it easier for you to get your bot up and running and doing its thing.” Well, those bots that you can make in the afternoon are very, very simple bots. They are question-answer bots. They are bots that aren’t necessarily bringing real value to businesses or consumers. For example, let’s say you contact your auto insurance company about a rock that came up from the road and cracked your windshield. A whole business process must be followed to address that. If you are going to build conversational AI around that conversation, you have to design it with the expertise required to craft a conversation around that business process. The buzz has made people think this is very easy, but it’s not as simple as plug and play. And it’s not something that just anyone can do. What we’re seeing with some companies is that they are trying to do this on their own. They’re saying, “Hey, this is so easy. We don’t need to hire the experts. We can build this on our own.” And what they are delivering are poor and unconnected experiences. And that makes the technology look bad because the reality with conversational AI is that the technology itself doesn’t provide the solution. You have to build the solution on top of the technology. The technology is just an enabler that requires an expert skill. We coined a term at Nuance called VUI (Voice User Interface) design, which is also sometimes called Conversation Experience design. Our people working on it have psychology backgrounds, literature backgrounds, linguistics backgrounds. We even have a handful of PhD’s in psycholinguistics, which is the psychology of language. (Yes, that field does exist.) So, the AI buzz can make people think this is super easy when the reality is that it’s an expert skill. We see two directions this has to go, both of which we are taking right now at Nuance. One is we must make that pro skill, that VUI design skill, more data-driven and require less effort. Right now, it’s a totally manual job where a VUI designer has to sit down with the subject-matter expert. Whatever we learn from the subject-matter expert gets written down into a conversation flow. That is reviewed, and then the system ultimately gets built. That is all a process involving humans. We’ve just announced Project Pathfinder. Already, we’ve done some proof-of-concept work with our customers, taking conversations happening today in the contact center between two humans, ingesting those and graphically exposing the entire call flow. We can auto-generate that dialog flow – the conversation between two humans – and use that as our basis for building out a bot. That’s an advancement that no one else has right now. We have advanced our technology over 20 years, and now we are turning our attention to the design process. Pathfinder is aimed at making the design process a lot more data-driven. Our second direction right now is to actually deliver on this promise of AI as a single brain that knows all about customers. Enterprises have huge volumes of data about their customers. They can leverage that data when they are having a conversation with a customer to make predictions with less effort. We are working on that key area, which we think will bring even more value to enterprises and their customers. What is making the biggest impact from a customer ROI perspective? Are there any industries where it’s really making a positive impact to the bottom line? Are there any notable customer deployments that you’d like to mention? A. The reason why there’s such an AI revolution now is that compute power and data are a lot cheaper. These technologies are now available to the masses, where previously they had only been available to large enterprises and some niche players that had the money and the volume to really use them. We have several customers that are saving a lot of money every day using AI technology. In our phone channel, we have large customers getting up to 30 million incoming phone calls a month. It’s the high-cost channel for all our customers. The traditional solution had been to offshore customer support, which tended to have a negative impact on customer satisfaction. Customers often want a human touch and to have their problem solved quickly and easily, and that’s where AI technology is really making a huge impact on the bottom line: by being able to understand and solve problems quickly and efficiently. We did a study with one customer that showed it got the highest satisfaction rating from use of self-service automation using AI technology. So, people may complain about using a bot, but when the bot has been well designed, they are delighted using it. Many companies today are offering conversational AI. Is there anything specifically Nuance is doing to stand out? The way we stand out is twofold. One, the technology itself doesn’t deliver the ROI. That’s the black-and-white reality. New entrants to the market today are coming with technology only and saying, “Hey, we have a better way of doing natural language understanding. We have a better way of doing speech recognition. We have a better way of doing these things.” And the reality is, many of them are doing things exactly the same way we’ve been doing it. Some are doing things slightly differently, but the reality is, that’s just technology. The way natural language is used in the enterprise space is mainly to get the intent: “What is it you’re contacting us for?” And that functionally solves the problem. The accuracy of recognizing an intent today is in the low- to mid-90 percentage range. Being able to get to that level of accuracy is really, really close to human capability. While achieving this accuracy does involve technology, it also requires a knowledge base and understanding. Therefore, while many of these folks are coming to market with their conversational AI technology, they’re bringing technology without the knowledge and expertise, and actually not solving the problem or request. At Nuance, we take pride in our technology and our large solutions team that builds conversational AI-based custom applications directly with our customers. We understand their business processes. We sit with them to bring our experts in healthcare, banking, airlines, etc. – whatever industry they are in. We bring in the VUI designers to help craft these intelligent conversation applications on top of AI technology. One of our big differentiators is that we’re not just selling our world-class technology (although there’s no one out there with differentiating technology above ours). We are also selling a solutions team that can actually deliver these solutions. The second major differentiator is that, because we’ve been in this space for so long, we are good at understanding the challenges, which extend well beyond technology. Project Pathfinder, for example, is taking in unstructured, unlabeled, conversational logs and creating a graph from them. No one else is doing that. No one else even knows they have to do that because they’re so new to the space. So, technology and solutions are keeping us ahead of the competition. Thank you Tom! For more information, go to Nuance. Read more »
  • Guide to Your Personal Data and Who Is Using It – From Wired
    ON THE INTERNET, the personal data users give away for free is transformed into a precious commodity. The puppy photos people upload train machines to be smarter. The questions they ask Google uncover humanity’s deepest prejudices. And their location histories tell investors which stores attract the most shoppers. Even seemingly benign activities, like staying in and watching a movie, generate mountains of information, treasure to be scooped up later by businesses of all kinds. Personal data is often compared to oil—it powers today’s most profitable corporations, just like fossil fuels energized those of the past. But the consumers it’s extracted from often know little about how much of their information is collected, who gets to look at it, and what it’s worth. Every day, hundreds of companies you may not even know exist gather facts about you, some more intimate than others. That information may then flow to academic researchers, hackers, law enforcement, and foreign nations—as well as plenty of companies trying to sell you stuff. What Constitutes “Personal Data”? The internet might seem like one big privacy nightmare, but don’t throw your smartphone out the window just yet. “Personal data” is a pretty vague umbrella term, and it helps to unpack exactly what it means. Health records, social security numbers, and banking details make up the most sensitive information stored online. Social media posts, location data, and search-engine queries may also be revealing but are also typically monetized in a way that, say, your credit card number is not. Other kinds of data collection fall into separate categories—ones that may surprise you. Did you know some companies are analyzingthe unique way you tap and fumble with your smartphone? All this information is collected on a wide spectrum of consent: Sometimes the data is forked over knowingly, while in other scenarios users might not understand they’re giving up anything at all. Often, it’s clear something is being collected, but the specifics are hidden from view or buried in hard-to-parse terms-of-service agreements. Consider what happens when someone sends a vial of saliva to 23andme. The person knows they’re sharing their DNA with a genomics company, but they may not realize it will be resold to pharmaceutical firms. Many apps use your location to serve up custom advertisements, but they don’t necessarily make it clear that a hedge fund may also buy that location data to analyze which retail stores you frequent. Anyone who has witnessed the same shoe advertisement follow them around the web knows they’re being tracked, but fewer people likely understand that companies may be recording not just their clicks but also the exact movements of their mouse. In each of these scenarios, the user received something in return for allowing a corporation to monetize their data. They got to learn about their genetic ancestry, use a mobile app, or browse the latest footwear trends from the comfort of their computer. This is the same sort of bargain Facebook and Google offer. Their core products, including Instagram, Messenger, Gmail, and Google Maps, don’t cost money. You pay with your personal data, which is used to target you with ads. Who Buys, Sells, and Barters My Personal Data? The trade-off between the data you give and the services you get may or may not be worth it, but another breed of business amasses, analyzes, and sells your information without giving you anything at all: data brokers. These firms compile info from publicly available sources like property records, marriage licenses, and court cases. They may also gather your medical records, browsing history, social media connections, and online purchases. Depending on where you live, data brokers might even purchase your information from the Department of Motor Vehicles. Don’t have a driver’s license? Retail stores sell info to data brokers, too. The information data brokers collect may be inaccurate or out of date. Still, it can be incredibly valuable to corporations, marketers, investors, and individuals. In fact, American companies alone are estimated to have spent over $19 billion in 2018 acquiring and analyzing consumer data, according to the Interactive Advertising Bureau. Data brokers are also valuable resources for abusers and stalkers. Doxing, the practice of publicly releasing someone’s personal information without their consent, is often made possible because of data brokers. While you can delete your Facebook account relatively easily, getting these firms to remove your information is time-consuming, complicated, and sometimes impossible. In fact, the process is so burdensome that you can pay a service to do it on your behalf. Amassing and selling your data like this is perfectly legal. While some states, including California and Vermont, have recently moved to put more restrictions on data brokers, they remain largely unregulated. The Fair Credit Reporting Act dictates how information collected for credit, employment, and insurance reasons may be used, but some data brokers have been caught skirting the law. In 2012 the “person lookup” site Spokeo settled with the FTC for $800,000 over charges that it violated the FCRA by advertising its products for purposes like job background checks. And data brokers that market themselves as being more akin to digital phone books don’t have to abide by the regulation in the first place. There are also few laws governing how social media companies may collect data about their users. In the United States, no modern federal privacy regulation exists, and the government can even legally request digital data held by companies without a warrant in many circumstances (though the Supreme Court recently expanded Fourth Amendment protections to a narrow type of location data). The good news is, the information you share online does contribute to the global store of useful knowledge: Researchers from a number of academic disciplines study social media posts and other user-generated data to learn more about humanity. In his book, Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are, Seth Stephens-Davidowitz argues there are many scenarios where humans are more honest with sites like Google than they are on traditional surveys. For example, he says, fewer than 20 percent of people admit they watch porn, but there are more Google searches for “porn” than “weather.” Personal data is also used by artificial intelligence researchers to train their automated programs. Every day, users around the globe upload billions of photos, videos, text posts, and audio clips to sites like YouTube, Facebook, Instagram, and Twitter. That media is then fed to machine learning algorithms, so they can learn to “see” what’s in a photograph or automatically determine whether a post violates Facebook’s hate-speech policy. Your selfies are literally making the robots smarter. Congratulations. Read the source article in Wired. Read more »
WordPress RSS Feed Retriever by Theme Mason

Artificial Intelligence Technology and the Law

  • Government Plans to Issue Technical Standards For Artificial Intelligence Technologies
    On February 11, 2019, the White House published a plan for developing and protecting artificial intelligence technologies in the United States, citing economic and national security concerns among other reasons for the action.  Coming two years after Beijing’s 2017 announcement that China intends to be the global leader in AI by 2030, President Trump’s Executive Order on Maintaining American Leadership in Artificial Intelligence lays out five principles for AI, including “development of appropriate technical standards and reduc[ing] barriers to the safe testing and deployment of AI technologies in order to enable the creation of new AI-related industries and the adoption of AI by today’s industries.”  The Executive Order, which lays out a framework for an “American AI Initiative” (AAII), tasks the White House’s National Science and Technology Council (NSTC) Select Committee on Artificial Intelligence, established in 2018, with identifying federal government agencies to develop and implement the technical standards (so-called “implementing agencies”). Unpacking the AAII’s technical standards principle suggests two things.  First, federal governance of AI under the Trump Administration will favor a policy and standards governance approach over a more onerous command-and-control-type regulatory agency rulemaking approach leading to regulations (which the Trump administration often refers to as “barriers”).  Second, no technical standards will be adopted that stand in the way of the development or use of AI technologies at the federal level if they impede economic and national security goals. So what sort of technical standards might the Select Committee on AI and the implementing agencies come up with?  And how might those standards impact government agencies, government contractors, and even private businesses from a legal perspective? The AAII is short on answers to those questions, and we won’t know more until at least August 2019 when the Secretary of Commerce, through the Director of the National Institute of Standards and Technology (NIST), is required by the AAII to issue a plan “for Federal engagement in the development of technical standards and related tools in support of reliable, robust, and trustworthy systems that use AI technologies.”  Even so, it is instructive to review some relevant technical standards and related legal issues in anticipation of what might lie ahead for the United States AI industry. A survey of technical standards used across a spectrum of different industries shows that they can take many different forms, but often they classify as prescriptive or performance-based.  Pre-determined prescriptive metrics may specify requirements for things like accuracy, quality, output, materials, composition, and consumption.  In the AI space, a prescriptive standard could involve a benchmark for classification accuracy (loss or error) using a standardized data set (i.e., how well does the system work), or a numerical upper limit on power consumption, latency, weight, and size.  Prescriptive standards can be one-size-fits-all, or they can vary. Performance-based standards describe practices (minimum, best, commercially reasonable, etc.) focusing on results to be achieved.  In many situations, the performance-based approach provides more flexibility compared to using prescriptive standards.  In the context of AI, a performance-based standard could require a computer vision system to detect all objects in a specified field of view, and tag and track them for a period of time.  How the developer achieves that result is less important in performance-based standards. Technical standards may also specify requirements for the completion of risk assessments to numerically compare an AI system’s expected benefits and impacts to various alternatives.  Compliance with technical standards may be judged by advisory committees who follow established procedures for independent and open review.  Procedures may be established for enforcement of technical standards when non-compliance is observed.  Depending on the circumstances, technical standards may be published for the public to see or they may be maintained in confidence (e.g., in the case of national security).  Technical standards are often reviewed on an on-going or periodic basis to assess the need for revisions to reflect changes in previous assumptions (important in cases when rapid technological improvements or shifts in priorities occur). Under the direction of the AAII, the White House’s Select Committee and various designated implementing agencies could develop new technical standards for AI technologies, but they could also adopt (and possibly modify) standards published by others.  The International Organization for Standards (ISO), American National Standards Institute (ANSI), National Institute of Standards and Technology (NIST), and the Institute for Electronics and Electrical Engineers (IEEE) are among the few private and public organizations that have developed or are developing AI standards or guidance.  Individual state legislatures, academic institutions, and tech companies have also published guidance, principles, and areas of concern that could be applicable to the development of technical and non-technical standards for AI technologies.  By way of example, the ISO’s technical standard for “big data” architecture includes use cases for deep learning applications and large scale unstructured data collection.  The Partnership on AI, a private non-profit organization whose board consists of representatives from IBM, Google, Microsoft, Apple, Facebook, Amazon, and others, has developed what it considers “best practices” for AI technologies. Under the AAII, the role of technical standards, in addition to helping build an AI industry, will be to “minimize vulnerability to attacks from malicious actors and reflect Federal priorities for innovation, public trust, and public confidence in systems that use AI technologies.”  It is hard to imagine a purely technical standard addressing trust and confidence, though a non-technical standards-setting process could address those issues by, for example, introducing measures related to fairness, accountability, and transparency.  Consider the example of delivering AI-based healthcare services at Veterans Administration facilities, where trust and confidence could be reflected in non-technical standards that provide for the publication of clear, understandable explanations about how an AI system works and how it made a decision that affected a patent’s care.  Addressing trust and confidence could also be reflected in requirements for open auditing of AI systems.  The IEEE’s “Ethically Aligned Design” reference considers these and related issues. Another challenge in developing technical standards is to avoid incorporating patented technologies “essential” to the standards adopted by the government, or if unavoidable, to develop rules for disclosure and licensing of essential patents.  As the court in Apple v. Motorola explained, “[s]ome technological standards incorporate patented technology. If a patent claims technology selected by a standards-setting organization, the patent is called an ‘essential patent.’ Many standards-setting organizations have adopted rules related to the disclosure and licensing of essential patents. The policies often require or encourage members of the organization to identify patents that are essential to a proposed standard and to agree to license their essential patents on fair, reasonable and nondiscriminatory terms to anyone who requests a license. (These terms are often referred to by the acronyms FRAND or RAND.)  Such rules help to insure that standards do not allow the owners of essential patents to abuse their market power to extort competitors or prevent them from entering the marketplace.”  See Apple, Inc. v. Motorola Mobility, Inc., 886 F. Supp. 2d 1061 (WD Wis. 2012).  Given the proliferation of new AI-related US patents issued to tech companies in recent years, the likelihood that government technical standards will encroach on some of those patents seems high. For government contractors, AI technical standards could be imposed on them through the government contracting process.  A contracting agency could incorporate new AI technical standards by reference in government contracts, and those standards would flow through to individual task and work orders performed by contractors under those contracts.  Thus, government contractors would need to review and understand the technical standards in the course of executing a written scope of work to ensure they are in compliance.  Sponsoring agencies would likely be expected to review contractor deliverables to measure compliance with applicable AI technical standards.  In the case of non-compliance, contracting officials and their sponsoring agency would be expected to deploy their enforcement authority to ensure problems are corrected, which could include monetary penalties assessed against contractors. Although private businesses (i.e., not government contractors) may not be directly affected by agency-specific technical standards developed under the AAII, customers of those private businesses could, absent other relevant or applicable technical standards, use the government’s AI technical standards as a benchmark when evaluating a business’s products and services.  Moreover, even if federal AI-based technical standards do not directly apply to private businesses, there is certainly the possibility that Congress could legislatively mandate the development of similar or different technical and non-technical standards and other requirements applicable to a business’ AI technologies sold and used in commerce. The president’s Executive Order on AI has turned an “if” into a “when” in the context of federal governance of AI technologies.  If you are a stakeholder, now is a good time to put resources into closely monitoring developments in this area to prepare for possible impacts. Read more »
  • Washington State Seeks to Root Out Bias in Artificial Intelligence Systems
    The harmful effects of biased algorithms have been widely reported.  Indeed, some of the world’s leading tech companies have been accused of producing applications, powered by artificial intelligence (AI) technologies, that were later discovered to exhibit certain racial, cultural, gender, and other biases.  Some of the anecdotes are quite alarming, to say the least.  And while not all AI applications have these problems, it only takes a few concrete examples before lawmakers begin to take notice. In New York City, lawmakers began addressing algorithmic bias in 2017 with the introduction of legislation aimed at eliminating bias from algorithmic-based automated decision systems used by city agencies.  That effort led to the establishment of a Task Force in 2018 under Mayor de Blasio’s office to examine the issue in detail.  A report from the Task Force is expected this year. At the federal level, an increased focus by lawmakers on algorithmic bias issues began in 2018, as reported previously on this website (link) and elsewhere.  Those efforts, by both House and Senate members, focused primarily on gathering information from federal agencies like the FTC, and issuing reports highlighting the bias problem.  Expect congressional hearings in the coming months. Now, Washington State lawmakers are addressing bias concerns.  In companion bills SB-5527 and HB-1655, introduced on January 23, 2019, lawmakers in Olympia drafted a rather comprehensive piece of legislation aimed at governing the use of automated decision systems by state agencies, including the use of automated decision-making in the triggering of automated weapon systems.  As many in the AI community have discussed, eliminating algorithmic-based bias requires consideration of fairness, accountability, and transparency, issues the Washington bills appear to address.  But the bills also have teeth, in the form of a private right of action allowing those harmed to sue. Although the aspirational language of legislation often only provides a cursory glimpse at how stakeholders might be affected under a future law, especially in those instances where, as here, an agency head is tasked with producing implementing regulations, an examination of automated decisions system legislation like Washington’s is useful if only to understand how  states and the federal government might choose to regulate aspects of AI technologies and their societal impacts. Purpose and need for anti-bias algorithm legislation According to the bills’ sponsors, in Washington, automated decision systems are rapidly being adopted to make or assist in core decisions in a variety of government and business functions, including criminal justice, health care, education, employment, public benefits, insurance, and commerce.  These systems, the lawmakers say, are often deployed without public knowledge and are unregulated.  Their use raises concerns about due process, fairness, accountability, and transparency, as well as other civil rights and liberties.  Moreover, reliance on automated decision systems without adequate transparency, oversight, or safeguards can undermine market predictability, harm consumers, and deny historically disadvantaged or vulnerable groups the full measure of their civil rights and liberties. Definitions, Prohibited Actions, and Risk Assessments The new Washington law would define “automated decision systems” as any algorithm, including one incorporating machine learning or other AI techniques, that uses data-based analytics to make or support government decisions, judgments, or conclusions.  The law would distinguish “automated final decision system,” which are systems that make “final” decisions, judgments, or conclusions without human intervention, and “automated support decision system,” which provide information to inform the final decision, judgment, or conclusion of a human decision maker. Under the new law, in using an automated decision system, an agency would be prohibited from discriminating against an individual, or treating an individual less favorably than another, in whole or in part, on the basis of one or more factors enumerated in RCW 49.60.010.  An agency would be outright prohibited from developing, procuring, or using an automated final decision system to make a decision impacting the constitutional or legal rights, duties, or privileges of any Washington resident, or to deploy or trigger any weapon. Both versions of the bill include lengthy provisions detailing algorithmic accountability reports that agencies would be required to produce and publish for public comment.  Among other things, these reports must include clear information about the type or types of data inputs that a technology uses; how that data is generated, collected, and processed; and the type or types of data the systems are reasonably likely to generate, which could help reveal the degree of bias inherent in a system’s black box model.  The accountability reports also must identify and provide data showing benefits; describe where, when, and how the technology is to be deployed; and identify if results will be shared with other agencies. An agency that deploys an approved report would then be required to follow conditions that are set forth in the report. Although an agency’s choice to classify its automated decision system as one that makes “final” or “support” decisions may be given deference by courts, the designations are likely to be challenged if the classification is not justified.  One reason a party might challenge designations is to obtain an injunction, which may be available in the case where an agency relies on a final decision made by an automated decision system, whereas an injunction may be more difficult to obtain in the case of algorithmic decisions that merely support a human decision-maker.  The distinction between the two designations may also be important during discovery, under a growing evidentiary theory of “machine testimony” that includes cross-examining machines witnesses by gaining access to source code and, in the case of machine learning models, the developer’s data used to train a machine’s model.  Supportive decision systems involving a human making a final decision may warrant a different approach to discovery. Conditions impacting software makers Under the proposed law, public agencies that use automated decision systems would be required to publicize the system’s name, its vendor, and the software version, along with the decision it will be used to make or support.  Notably, a vendor must make its software and the data used in the software “freely available” before, during, and after deployment for agency or independent third-party testing, auditing, or research to understand its impacts, including potential bias, inaccuracy, or disparate impacts.  The law would require any procurement contract for an automated decision system entered into by a public agency to include provisions that require vendors to waive any legal claims that may impair the “freely available” requirement.  For example, contracts with vendors could not contain nondisclosure impairment provisions, such as those related to assertions of trade secrets. Accordingly, software companies who make automated decision systems will face the prospect of waiving proprietary and trade secret rights and opening up their algorithms and data to scrutiny by agencies, third parties, and researchers (presumably, under terms of confidentiality).  If litigation were to ensue, it could be difficult for vendors to resist third-party discovery requests on the basis of trade secrets, especially if information about auditing of the system by the state agency and third-party testers/researchers is available through administrative information disclosure laws.  A vendor who chooses to reveal the inner workings of a black box software application without safeguards should consider at least financial, legal, and market risks associated with such disclosure. Contesting automated decisions and private right of action Under the proposed law, public agencies would be required to announce procedures how an individual impacted by a decision made by an automated decision system can contest the decision.  In particular, any decision made or informed by an automated decision system will be subject to administrative appeal, an immediate suspension if a legal right, duty, or privilege is impacted by the decision, and a potential reversal by a human decision-maker through an open due process procedure.  The agency must also explain the basis for its decision to any impacted individual in terms “understandable” to laypersons including, without limitation, by requiring the software vendor to create such an explanation.  Thus, vendors may become material participants in administrative proceedings involving a contested decision made by its software. In addition to administrative relief, the law would provide a private right of action for injured parties to sue public agencies in state court.  In particular, any person who is injured by a material violation of the law, including denial of any government benefit on the basis of an automated decision system that does not meet the standards of the law, may seek injunctive relief, including restoration of the government benefit in question, declaratory relief, or a writ of mandate to enforce the law. For litigators representing injured parties in such cases, dealing with evidentiary issues involving information produced by machines would likely follow Washington judicial precedent in areas of administrative law, contracts, tort, civil rights, the substantive law involving the agency’s jurisdiction (e.g., housing, law enforcement, etc.), and even product liability.  In the case of AI-based automated decision systems, however, special attention may need to be given to the nuances of machine learning algorithms to prepare experts and take depositions in cases brought under the law.  Although the aforementioned algorithmic accountability report could be useful evidence for both sides in an automated decision system lawsuit, merely understanding the result of an algorithmic decision may not be sufficient when assessing if a public agency was thorough in its approach to vetting a system.  Being able to describe how the automated decision system works will be important.  For agencies, understanding the nuances of the software products they procure will be important to establish that they met their duty to vet the software under the new law. For example, where AI machine learning models are involved, new data, or even previous data used in a different way (i.e., a different cross-validation scheme or a random splitting of data into new training and testing subsets), can generate models that produce slightly different outcomes.  While small, the difference could mean granting or denying agency services to constituents.  Moreover, with new data and model updates comes the possibility of introducing or amplifying bias that was not previously observed.  The Washington bills do not appear to include provisions imposing an on-going duty on vendors to inform agencies when bias or other problems later appear in software updates (though it’s possible the third party auditors or researchers noted above might discover it).  Thus, vendors might expect agencies to demand transparency as a condition set forth in acquisition agreements, including software support requirements and help with developing algorithmic accountability reports.  Vendors might also expect to play a role in defending against claims by those alleging injury, should the law pass.  And they could be asked to shoulder some of the liability either through indemnification or other means of contractual risk-shifting to the extent the bills add damages as a remedy. Read more »
  • What’s in a Name? A Chatbot Given a Human Name is Still Just an Algorithm
    Due in part to the learned nature of artificial intelligence technologies, the spectrum of things that exhibit “intelligence” has, in debates over such things, expanded to include certain advanced AI systems.  If a computer vision system can “learn” to recognize real objects and make decisions, the argument goes, its ability to do so can be compared to that of humans and thus should not be excluded from the intelligence debate.  By extension, AI systems that can exhibit intelligence traits should not be treated like mere goods and services, and thus laws applicable to such good and services ought not to apply to them. In some ways, the marketing of AI products and services using names commonly associated with humans, such as “Alexa,” “Sophia,” and “Siri,” buttresses the argument that laws applicable to non-human things should not strictly apply to AI.  For now, however, lawmakers and the courts struggling with practical questions about regulating AI technologies can justifiably apply traditional goods and services laws to named AI systems just as they do to non-named system.  After all, a robot or chatbot doesn’t become more humanlike and less like a man-made product merely because it’s been anthropomorphized.  Even so, when future technological breakthroughs suggest artificial general intelligence (AGI) is on the horizon, lawmakers and the courts will be faced with the challenge of amending laws to account for the differences between AGI systems and today’s narrow AI and other “unintelligent” goods and services.  For now, it’s instructive to consider why the rise in the use of names for AI system is not a good basis for triggering greater attention by lawmakers.  Indeed, as suggested below, other characteristics of AI system may be more useful in deciding when laws need to be amended.  To begin, the recent case of a chatbot named “Erica” is presented. The birth of a new bot In 2016, machine learning developers at Bank of America created a “virtual financial assistant” application called “Erica” (derived from the bank’s name America).  After conducting a search of existing uses of the name Erica in other commercial endeavors, and finding none in connection with a chatbot like theirs, BoA sought federal trademark protection for the ERICA mark in October 2016.  The US Patent and Trademark Office concurred with BoA’s assessment of prior uses and registered the mark on July 31, 2018.  Trademarks are issued in connection with actual uses of words, phrases, and logos in commerce, and in the case of BoA, the ERICA trademark was registered in connection with computer financial software, banking and financial services, and personal assistant software in banking and financial SaaS (software as a service).  The Erica app is currently described as possessing the utility to answer customer questions and make banking easier.  During its launch, BoA used the “she” pronoun when describing the app’s AI and predictive analytics capabilities, ostensibly because the name Erica is a stereotypical female gender name, but also because of the apparent female-sounding voice the app outputs as part of its human-bot interface. One of the existing uses of an Erica-like mark identified by BoA was an instance of “E.R.I.C.A,” which appeared in October 2010 when Erik Underwood, a Colorado resident, filed a Georgia trademark registration application for “E.R.I.C.A. (Electronic Repetitious Informational Clone Application).”  See Underwood v. Bank of Am., slip op., No. 18-cv-02329-PAB-MEH (D. Colo. Dec. 19, 2018).  On his application, Mr. Underwood described E.R.I.C.A. as “a multinational computer animated woman that has slanted blue eyes and full lips”; he also attached a graphic image of E.R.I.C.A. to his application.  Mr. Underwood later sought a federal trademark application (filed in September 2018) for an ERICA trademark (without the separating periods).  At the time of his lawsuit, his only use of E.R.I.C.A. was on a searchable movie database website. In May 2018, Mr. Underwood sent a cease-and-desist letter to BoA regarding BoA’s use of Erica, and then filed a lawsuit in September 2018 against the bank alleging several causes of action, including “false association” under § 43(a) of the Lanham Act, 15 U.S.C. § 1125(a)(1)(A).  Section 43(a) states, in relevant part, that any person who, on or in connection with any goods or services, uses in commerce a name or a false designation of origin which is likely to cause confusion, or to cause mistake, or to deceive as to the affiliation, connection, or association of such person with another person, or as to the origin, sponsorship, or approval of his or her goods, services, or commercial activities by another person, shall be liable in a civil action by a person who believes that he or she is likely to be damaged by such act.  In testimony, Mr. Underwood stated that the E.R.I.C.A. service mark was being used in connection with “verbally tell[ing] the news and current events through cell phone[s] and computer applications” and he described plans to apply an artificial intelligence technology to E.R.I.C.A.  Mr. Underwood requested the court enter a preliminary injunction requiring BoA to cease using the Erica name. Upon considering the relevant preliminary injunction factors and applicable law, the District Court denied Mr. Underwood’s request for an injunction on several grounds, including the lack of relevant uses of E.R.I.C.A. in the same classes of goods and services that BoA’s Erica was being used in. Giving AI a persona may boost its economic value and market acceptance Not surprisingly, the District Court’s preliminary injunction analysis rested entirely on perception and treatment of the Erica and E.R.I.C.A. systems as nothing more than services, something neither party disputed or challenged.  Indeed, each party’s case-in-chief depended on their convincing the court that their applications fit squarely in the definition of goods and services despite the human-sounding names they chose to attach to them.  The court’s analysis, then, illuminated one of the public policies underlying laws like the Lanham Act, which is the protection of the economic benefits associated with goods and services created by people and companies.  The name Erica provides added economic value to each party’s creation and is an intangible asset associated with their commercial activities. The use of names has long been found to provide value to creators and owners, and not just in the realm of hardware and software.  Fictional characters like “Harry Potter,” which are protected under copyright and trademark laws, can be intellectual assets having tremendous economic value.  Likewise, namesake names carried over to goods and services, like IBM’s “Watson”–named after the company’s first CEO, John Watson–provide real economic benefits that might not have been achieved without a name, or even with a different name.  In the case of humanoid robots, like Hanson Robotics’ “Sophia,” which is endowed with aspects of AI technologies and was reportedly granted “citizenship” status in Saudi Arabia, certain perceived and real economic value is created by distinguishing the system from all other robots by using a real name (as compared to, for example, a simple numerical designation). On the other end of the spectrum are names chosen for humans, the uses of which are generally unrestricted from a legal perspective.  Thus, naming one’s baby “Erica” or even “Harry Potter” shouldn’t land a new parent in hot water.  At the same time, those parents aren’t able to stop others from using the same names for other children.  Although famous people may be able to prevent others from using their names (and likenesses) for commercial purposes, the law only recognizes those situations when the economic value of the name or likeness is established (though demonstrating economic value is not always necessary under some state right of publicity laws).  Some courts have gone so far as to liken the right to protect famous personas to a type of trademark in a person’s name because of the economic benefits attached to it, much the same way a company name, product name, or logo attached to a product or service can add value. Futurists might ask whether a robot or chatbot demonstrating a degree of intelligence and that endowed with unique human-like traits, including a unique persona (e.g., name and face generated from a generative-adversarial network) and the ability to recognize and respond to emotions (e.g., using facial coding algorithms in connection with a human-robot interface), thus making them sufficiently differentiable from all other robots and chatbots (at least superficially), should have special treatment.  So far, endowing AI technologies with a human form, gender, and/or a name has not motivated lawmakers and policymakers to pass new laws aimed at regulating AI technologies.  Indeed, lawmakers and regulators have so far proposed, and in some cases passed, laws and regulations placing restrictions on AI technologies based primarily on their specific applications (uses) and results (impacts on society).  For example, lawmakers are focusing on bot-generated spread and amplification of disinformation on social media, law enforcement use of facial recognition, the private business collection and use of face scans, users of drones and highly automated vehicles in the wild, production of “deepfake” videos, the harms caused by bias in algorithms, and others.  This application/results-focused approach, which acknowledges explicitly or implicitly certain normative standards or criteria for acceptable actions, as a means to regulate AI technology is consistent with how lawmakers have treated other technologies in the past. Thus, marketers, developers, and producers of AI systems who personify their chatbots and robots may sleep well knowing their efforts may add value to their creations and alter customer acceptance and attitudes about their AI systems, but they are unlikely to cause lawmakers to suddenly consider regulating them. At some point, however, advanced AI systems will need to be characterized in some normative way if they are to be governed as a new class of things.  The use of names, personal pronouns, personas, and metaphors associating bots to humans may frame bot technology in a way that ascribes particular values and norms to it (Jones 2017).  These might include characteristics such as utility, usefulness (including positive benefits to society), adaptability, enjoyment, sociability, companionship, and perceived or real “behavioral” control, which some argue are important in evaluating user acceptance of social robots.  Perhaps these and other factors, in addition to some measure of intelligence, need to be considered when deciding if an advanced AI bot or chatbot should be treated under the law as something other than a mere good or service.  The subjective nature of those factors, however, would obviously make it challenging to create legally-sound definitions of AI for governance purposes.  Of course, laws don’t have to be precise (and sometimes they are intentionally written without precision to provide flexibility in their application and interpretation), but a vague law won’t help an AI developer or marketer know whether his or her actions and products are subject to an AI law.  Identifying whether to treat bots as goods and services or as something else deserving of a different set of regulations, like those applicable to humans, is likely to involve a suite of factors that permit classifying advanced AI on the spectrum somewhere between goods/services and humans. Recommended reading  The Oxford Handbook of Law, Regulation, and Technology is one of my go-to references for timely insight about topics discussed on this website.  In the case of this post, I drew inspiration from Chapter 25: Hacking Metaphors in the Anticipatory Governance of Emerging Technology: The Case of Regulating Robots, by Meg Leta Jones and Jason Millar. Read more »
  • The Role of Explainable Artificial Intelligence in Patent Law
    Although the notion of “explainable artificial intelligence” (AI) has been suggested as a necessary component of governing AI technology, at least for the reason that transparency leads to trust and better management of AI systems in the wild, one area of US law already places a burden on AI developers and producers to explain how their AI technology works: patent law.  Patent law’s focus on how AI systems work was not borne from a Congressional mandate. Rather, the Supreme Court gets all the credit–or blame, as some might contend–for this legal development, which began with the Court’s 2014 decision in Alice Corp. Pty Ltd. v. CLS Bank International. Alice established the legal framework for assessing whether an invention fits in one of the patent law’s patent-eligible categories (i.e., any “new and useful process, machine, manufacture, or composition of matter” or improvements thereof) or is a patent-ineligible concept (i.e., law of nature, natural phenomenon, or abstract idea).  Alice Corp. Pty Ltd. v. CLS Bank International, 134 S. Ct. 2347, 2354–55 (2014); 35 USC § 101. Understanding how the idea of “explaining AI” came to be following Alice, one must look at the very nature of AI technology.  At their core, AI systems based on machine learning models generally transform input data into actionable output data, a process US courts and the Patent Office have historically found to be patent-ineligible.  Consider a decision by the US Court of Appeals for the Federal Circuit, whose judges are selected for their technical acumen as much as for their understanding of the nuances of patent and other areas of law, that issued around the same time as Alice: “a process that employs mathematical algorithms to manipulate existing information to generate additional information is not patent eligible.”  Digitech Image Techs, LLC v. Elecs. v. Imaging, Inc., 758 F.3d 1344, 1351 (Fed. Cir. 2014).  While Alice did not specifically address AI or mandate anything resembling explainable AI, it nevertheless spawned a progeny of Federal Circuit, district court, and Patent Office decisions that did just that.  Notably, those decisions arose not because of notions that individuals impacted by AI algorithmic decisions ought to have the right to understand how those decisions were made or why certain AI actions were taken, but because explaining how AI systems works helps satisfy the quid pro quo that is fundamental to patent law: an inventor who discloses to the world details of what she has invented is entitled to a limited legal monopoly on her creation (provided, of course, the invention is patentable). The Rise of Algorithmic Scrutiny Alice arrived not long after Congress passed patent reform legislation called the America Invents Act (AIA) of 2011, provisions of which came into effect in 2012 and 2013.  In part, the AIA targeted a decade of what many consider a time of abusive patent litigation brought against some of the largest tech companies in the world and thousands of mom-and-pop and small business owners who were sued for doing anything computer-related.  This litigious period saw the term “patent troll” used more often to describe patent assertion companies that bought up dot-com-era patents covering the very basics of using the Internet and computerized business methods and then sued to collect royalties for alleged infringement. Not surprisingly, some of the same big tech companies that pushed for patent reform provisions now in the AIA to curb patent litigation in the field of computer technology also filed amicus curiae briefs in the Alice case to further weaken software patents.  The Supreme Court’s unanimous decision in Alice helped curtail troll-led litigation by formalizing a procedure, one that lower court judges could easily adopt, for excluding certain software-related inventions from the list of inventions that are patentable. Under Alice, a patent claim–the language used by inventors to describe what he or she claims to be his or her invention–falls outside § 101 when it is “directed to” one of the patent-ineligible concepts noted above.  If so, Alice requires consideration of whether the particular elements of the claim, evaluated “both individually and ‘as an ordered combination,'” add enough to “‘transform the nature of the claim'” into one of the patent-eligible categories.  Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350, 1353 (Fed.Cir. 2016) (quoting Alice, 134 S. Ct. at 2355).  While simple in theory, it took years of court and Patent Office decisions to explain how that 2-part test is to be employed, and only more recently how it applies to AI technologies.  Today, the Patent Office and courts across the US routinely find that algorithms are abstract (even though algorithms, including certain mental processes embodied in algorithmic form performed by a computer, are by most measures useful processes).  According to the Federal Circuit, algorithmic-based data collection, manipulation, and communication–functions most AI algorithms perform–are abstract. Artificial Intelligence, Meet Alice In a bit of ironic foreshadowing, the Supreme Court issued Alice in the same year that major advances in AI technologies were being announced, such as Google’s deep neural network architecture that prevailed in the 2014 ImageNet challenge (ILSVCR) and Ian Goodfellow’s generative adversarial network (GAN) model, both of which were major contributions to the field of computer vision. Even as more breakthroughs were being announced, US courts and the Patent Office began issuing Alice decisions regarding AI technologies and explaining why it’s crucial for inventors to explain how their AI inventions work to satisfy the second half of Alice’s 2-part test. In Purepredictive, Inc. v. H2O.AI, Inc., for example, the US District Court for the Northern District of California considered the claims of US Patent 8,880,446, which, according to the patent’s owner, involves “AI driving machine learning ensembling.”  The district court characterized the patent as being directed to a software method that performs “predictive analytics” in three steps.  Purepredictive, Inc. v. H2O.AI, Inc., slip op., No. 17-cv-03049-WHO (N.D. Cal. Aug. 29, 2017).  In the method’s first step, it receives data and generates “learned functions,” or, for example, regressions from that data. Second, it evaluates the effectiveness of those learned functions at making accurate predictions based on the test data. Finally, it selects the most effective learned functions and creates a rule set for additional data input. The court found the claims invalid on the grounds that they “are directed to the abstract concept of the manipulation of mathematical functions and make use of computers only as tools, rather than provide a specific improvement on a computer-related technology.” The claimed method, the district court said, is merely “directed to a mental process” performed by a computer, and “the abstract concept of using mathematical algorithms to perform predictive analytics” by collecting and analyzing information.  The court explained that the claims “are mathematical processes that not only could be performed by humans but also go to the general abstract concept of predictive analytics rather than any specific application.” In Ex Parte Lyren, the Patent Office’s Appeals Board, made up of three administrative law judges, rejected a claim directed to customizing video on a computer as being abstract and thus not patent-eligible.  In doing so, the board disagreed with the inventor, who argued the claimed computer system, which generated and displayed a customized video by evaluating a user’s intention to purchase a product and information in the user’s profile, was an improvement in the technical field of generating videos. The claimed customized video, the Board found, could be any video modified in any way.  That is, the rejected claims were not directed to the details of how the video was modified, but rather to the result of modifying the video.  Citing precedent, the board reiterated that “[i]n applying the principles emerging from the developing body of law on abstract ideas under section 101, … claims that are ‘so result-focused, so functional, as to effectively cover any solution to an identified problem’ are frequently held ineligible under section 101.”  Ex ParteLyren, No. 2016-008571 (PTAB, June 25, 2018) (citing Affinity Labs of Texas,LLC v. DirecTV, LLC, 838 F.3d 1253, 1265 (Fed. Cir. 2016) (quoting Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350, 1356 (Fed. Cir, 2016)); see also Ex parte Colcernian et al., No. 2018-002705 (PTAB, Oct. 1, 2018) (rejecting claims that use result-oriented language as not reciting the specificity necessary to show how the claimed computer processor’s operations differ from prior human methods, and thus are not directed to a technological improvement but rather are directed to an abstract idea). Notably, the claims in Ex Parte Lyren were also initially rejected as failing to satisfy a different patentability test–the written description requirement.  35 USC § 112.  In rejecting the claims as lacking sufficient description of the invention, the Patent Office Examiner found that the algorithmic features of the inventor’s claim were “all implemented inside a computer, and therefore all require artificial intelligence [(AI)] at some level” and thus require extensive implementation details “subject of cutting-edge research, e.g.[,] natural language processing and autonomous software agents exhibiting intelligent behavior.” The Examiner concluded that “one skilled in the art would not be persuaded that Applicant possessed the invention” because “it is not readily apparent how to make a device [to] analyze natural language.”  The Appeals Board disagreed and sided with the inventor who argued that his invention description was comprehensive and went beyond just artificial intelligence implementations.  Thus, while the description of how the invention worked was sufficiently set forth, Lyren’s claims focused too much on the results or application of the technology and thus were found to be abstract. In Ex Parte Homere, claims directed to “a computer-implemented method” involving “establishing a communication session between a user of a computer-implemented marketplace and a computer-implemented conversational agent associated with the market-place that is designed to simulate a conversation with the user to gather listing information, the Appeals Board affirmed an Examiner’s rejection of the claims as being abstract.  Ex Parte Homere, Appeal No. 2016-003447 (PTAB Mar. 29, 2018).  In doing so, the Appeals Board noted that the inventor had not identified anything in the claim or in the written description that would suggest the computer-related elements of the claimed invention represent anything more than “routine and conventional” technologies.  The most advanced technologies alluded to, the Board found, seemed to be embodiments in which “a program implementing a conversational agent may use other principles, including complex trained Artificial Intelligence (AI) algorithms.”  However, the claimed conversational agent was not so limited.  Instead, the Board concluded that the claims were directed to merely using recited computer-related elements to implement the underlying abstract idea, rather than being limited to any particular advances in the computer-related elements. In Ex Parte Hamilton, a rejection of a claim directed to “a method of planning and paying for advertisements in a virtual universe (VU), comprising…determining, via the analysis module, a set of agents controlled by an Artificial Intelligence…,” was affirmed as being patent ineligible.  Ex Parte Hamilton et al., Appeal No.2017-008577 (PTAB Nov. 20, 2018).  The Appeals Board found that the “determining” step was insufficient to transform the abstract idea of planning and paying for advertisements into patent-eligible subject matter because the step represented an insignificant data-gathering step and thus added nothing of practical significance to the underlying abstract idea. In Ex Parte Pizzorno, the Appeals Board affirmed a rejection of a claim directed to “a computer implemented method useful for improving artificial intelligence technology” as abstract.  Ex Parte Pizzorno, Appeal No. 2017-002355 (PTAB Sep. 21, 2018).  In doing so, the Board determined that the claim was directed to the concept of using stored health care information for a user to generate personalized health care recommendations based on Bayesian probabilities, which the Board said involved “organizing human activities and an idea in itself, and is an abstract idea beyond the scope of § 101.”  Considering each of the claim elements in turn, the Board also found that the function performed by the computer system at each step of the process was purely conventional in that each step did nothing more than require a generic computer to perform a generic computer function. Finally, in Ex Parte McAfee, the Appeals Board affirmed a rejection of a claim on the basis that it was “directed to the abstract idea of receiving, analyzing, and transmitting data.”  Ex Parte McAfee, Appeal No. 2016-006896 (PTAB May 22, 2018).  At issue was a method that included “estimating, by the ad service circuitry, a probability of a desired user event from the received user information, and the estimate of the probability of the desired user event incorporating artificial intelligence configured to learn from historical browsing information in the received user information, the desired user event including at least one of a conversion or a click-through, and the artificial intelligence including regression modeling.”  In affirming the rejection, the Board found that the functions performed by the computer at each step of the claimed process was purely conventional and did not transform the abstract method into a patent-eligible one. In particular, the step of estimating the probability of the desired user event incorporating artificial intelligence was found to be merely “a recitation of factors to be somehow incorporated, which is aspirational rather than functional and does not narrow the manner of incorporation, so it may include no more than incorporating results from some artificial intelligence outside the scope of the recited steps.” The above and other Alice decisions have led to a few general legal axioms, such as: a claim for a new abstract idea is still an abstract idea; a claim for a beneficial abstract idea is still an abstract idea; abstract ideas do not become patent-eligible because they are new ideas, are not previously well known, and are not routine activity; and, the “mere automation of manual processes using generic computers does not constitute a patentable improvement in computer technology.”  Synopsys, Inc. v. Mentor Graphics Corp., 839 F.3d 1138, 1151 (Fed. Cir. 2016); Ariosa Diagnostics, Inc. v. Sequenom, Inc., 788 F.3d 1371, 1379-80 (Fed. Cir. 2015); Ultramercial, Inc. v. Hulu, LLC, 772 F.3d. 709, 715-16 (Fed. Cir. 2014); Credit Acceptance Corp. v. Westlake Servs., 859 F.3d 1044, 1055 (Fed. Cir. 2017); see also SAP Am., Inc. v. Investpic, LLC, slip op. No. 2017-2081, 2018 WL2207254, at *2, 4-5 (Fed. Cir. May 15, 2018) (finding financial software patent claims abstract because they were directed to “nothing but a series of mathematical calculations based on selected information and the presentation of the results of those calculations (in the plot of a probability distribution function)”); but see Apple, Inc. v.Ameranth, Inc., 842 F.3d 1229, 1241 (Fed. Cir. 2016) (noting that “[t]he Supreme Court has recognized that all inventions embody, use,reflect, rest upon, or apply laws of nature, natural phenomena, or abstractideas[ ] but not all claims are directed to an abstract idea.”). The Focus on How, not the Results Following Alice, patent claims directed to an AI technology must recite features of the algorithm-based system that represent how the algorithm improves a computer-related technology and is not previously well-understood, routine, and conventional.  In PurePredictive, for example, the Northern California district court, which sees many software-related cases due to its proximity to the Bay Area and Silicon Valley, found that the claims of a machine learning ensemble invention were not directed to an invention that “provide[s] a specific improvement on a computer-related technology.”  See also Neochloris, Inc. v. Emerson Process Mgmt LLLP, 140 F. Supp. 3d 763, 773 (N.D. Ill. 2015) (explaining that patent claims including “an artificial neural network module” were invalid under § 101 because neural network modules were described as no more than “a central processing unit – a basic computer’s brain”). Satisfying Alice, thus, requires claims focusing on a narrow application of how an AI algorithmic model works, rather than the broader and result-oriented nature of what the model is used for.  This is necessary where the idea behind the algorithm itself could be used to achieve many different results.  For example, a claim directed to a mathematical process (even one that is said to be “computer-implemented”), and that could be performed by humans (even if it takes a long time), and that is directed to a result achieved instead of a specific application, will seemingly be patent-ineligible under today’s Alice legal framework. To illustrate, consider an image classification system, one that is based on a convolutional neural network.  Such a system may be patentable if the claimed system improves the field of computer vision technology. Claiming the invention in terms of how the elements of the computer are technically improved by its deep learning architecture and algorithm, rather than simply claiming a deep learning model using results-oriented language, may survive an Alice challenge, provided the claim does not merely cover an automated process that human used to do.  Moreover, claims directed to the multiple hidden layers, convolutions, recurrent connections, hyperperameters, and weights could also be claimed. By way of another example, a claim reciting “a computer-implemented process using artificial intelligence to generate an image of a person,” is likely abstract if it does not explain how the image is generated and merely claims a computerized process a human could perform.  But a claim that describes a unique AI system that specifies how it generates the image, including the details of a generative adversarial network architecture and its various inputs provided by physical devices (not routine data collection), its connections and hyperparameters, has a better chance of passing muster (keeping in mind, this only addresses the question of whether the claimed invention is eligible to be patented, not whether it is, in fact, patentable, which is an entirely different analysis and requires comparing the claim to prior art). Uncertainty Remains Although the issue of explaining how an AI system works in the context of patent law is still in flux, the number of US patents issued by the Patent Office mentioning “machine learning,” or the broader term “artificial intelligence,” has jumped in recent years. Just this year alone, US machine learning patents are up 27% compared to the same year-to-date period in 2017 (thru the end of November), according to available Patent Office records.  Even if machine learning is not the focus of many of them, the annual upward trend in patenting AI over the last several years appears unmistakable. But with so many patents invoking AI concepts being issued, questions about their validity may arise.  As the Federal Circuit has stated, “great uncertainty yet remains” when it comes to the test for deciding whether an invention like AI is patent-eligible under Alice, this despite the large number of cases that have “attempted to provide practical guidance.”  Smart Systems Innovations, LLC v. Chicago Transit Authority, slip. op. No. 2016-1233 (Fed. Cir. Oct. 18, 2017).  Calling the uncertainty “dangerous” for some of today’s “most important inventions in computing,” specifically mentioning AI, the Federal Circuit expressed concern that perhaps the application of the Alice test has gone too far, a concern mirrored in testimony by Andrei Iancu, Director of the Patent Office, before Congress in April 2018 (stating, in response to Judiciary Committee questions, that Alice and its progeny have introduced a degree of uncertainty into the area of subject matter eligibility, particularly as it relates to medical diagnostics and software-related inventions, and that Alice could be having a negative impact on innovation). Absent legislative changes abolishing or altering Alice, a solution to the uncertainty problem, at least in the context of AI technologies, lies in clarifying existing decisions issued by the Patent Office and courts, including the decisions summarized above.  While it can be challenging to explain why an AI algorithm made a particular decision or took a specific action (due to the black box nature of such algorithms once they are fully trained), it is generally not difficult to describe the structure of a deep learning or machine learning algorithm or how it works. Even so, it remains unclear whether and to what extent fully describing how one’s AI technology and including “how” features in patent claims will ever be sufficient to “add[] enough to transform the nature of an abstract algorithm into a patent-eligible [useful process].” If explaining how AI works is to have a future meaningful role in patent law, the courts or Congress will need to provide clarity. Read more »
  • California Appeals Court Denies Defendant Access to Algorithm That Contributed Evidence to His Conviction
    One of the concerns expressed by those studying algorithmic decision-making is the apparent lack of transparency. Those impacted by adverse algorithmic decisions often seek transparency to better understand the basis for the decisions. In the case of software used in legal proceedings, parties who seek explanations about software face a number of obstacles, including those imposed by evidentiary rules, criminal or civil procedural rules, and by software companies that resist discovery requests. The closely-followed issue of algorithmic transparency was recently considered by a California appellate court in People v. Superior Court of San Diego County, slip op. Case D073943 (Cal. App. 4th October 17, 2018), in which the People sought relief from a discovery order requiring the production of software and source code used in the conviction of Florencio Jose Dominguez. Following a hearing and review of the record and amicus briefs in support of Dominguez filed by the American Civil Liberties Union, the American Civil Liberties Union of San Diego and Imperial Counties, the Innocence Project, Inc., the California Innocence Project, the Northern California Innocence Project at Santa Clara University School of Law, Loyola Law School’s Project for the Innocent, and the Legal Aid Society of New York City, the appeals court granted the People’s relief. In doing so, the court considered, but was not persuaded by, the defense team’s “black box” and “machine testimony” arguments. At issue on appeal was Dominguez’s motion to compel production of a DNA testing program called STRmix used by local prosecutors in their analysis of forensic evidence (specifically, DNA found on the inside of gloves). STRmix is a “probabilistic genotyping” program that expresses a match between a suspect and DNA evidence in terms the probability of a match compared to a coincidental match. Probabilistic genotyping is said to reduce subjectivity in the analysis of DNA typing results. Dominguez’s counsel moved the trial court for an order compelling the People to produce the STRmix software program and related updates as well as its source code, arguing that defendant had a right to look inside the software’s “black box.” The trial court granted the motion and the People sought writ relief from the appellate court. On appeal, the appellate court noted that “computer software programs are written in specialized languages called source code” and “source code, which humans can read, is then translated into [a] language that computers can read.” Cadence Design Systems, Inc. v. Avant! Corp., 29 Cal. 4th 215, 218 at fn.3 (2002). The lab that used STRmix testified that it had no way to access the source code, which it licensed from a software authorized seller.  Thus,  the court considered whether the company that created the software should produce it. In concluding that the company was not obligated to produce the software and source code, the court, citing precedent, found that the company would have had no knowledge of the case but for the defendant’s  subpoena duces tecum, and it did not act as part of the prosecutorial team such that it was obligated to turn over exculpatory evidence (assuming software itself is exculpatory, which the court was reluctant to find). With regard to the defense team’s “black box” argument, the appellate court found nothing in the record to indicate that the STRmix software suffered a problem, as the defense team argued, that might have affected its results. Calling this allegation speculative, the court concluded that the “black box” nature of STRmix was not itself sufficient to warrant its production. Moreover, the court was unpersuaded by the defense team’s argument that the STRmix program essentially usurped the lab analyst’s role in providing the final statistical comparison, and so the software program—not the analyst using the software—was effectively the source of the expert opinion rendered at trial. The lab, the defense argued, merely acted in a scrivener’s capacity for STRmix’s analysis, and since the machine was providing testimony, Dominguez should be able to evaluate the software to defend against the prosecution’s case against him. The appellate court disagreed. While acknowledging the “creativity” of the defense team’s “machine testimony” argument (which relied heavily on Berkeley law professor Andrea Roth’s “Machine Testimony” article (126 Yale L.J. 1972 (2017)), the panel noted the testimony that STRmix did not act alone, that there were humans in the loop: “[t]here are still decisions that an analyst has to make on the front end in terms of determining the number of contributors to a particular sample and determin[ing] which peaks are from DNA or from potentially artifacts” and that the program then performs a “robust breakdown of the DNA samples,” based at least in part on “parameters [the lab] set during validation.” Moreover, after STRmix renders “the diagnostics,” the lab “evaluate[s] … the genotype combinations … . to see if that makes sense, given the data [it’s] looking at.” After the lab “determine[s] that all of the diagnostics indicate that the STRmix run has finished appropriately,” it can then “make comparisons to any person of interest or … database that [it’s] looking at.” While the appellate court’s decision mostly followed precedent and established procedure, it could easily have gone the other way and affirmed the trial judge’s decision granting Defendant’s motion to compel the STRmix software and source code, which would have given Dominguez better insight into the nature of the software’s algorithms, its parameters and limitations in view of validation studies, and the various possible outputs the model could have produced given a set of inputs. In particular, the court might have affirmed the trial judge’s decision to grant access to the STRmix software if the policy of imposing transparency in STRmix’s algorithmic decisions were given more consideration from the perspective of actual harm that might occur if software and source code are produced. Here, the source code owner’s objection to production was based in part on trade secret and other confidentiality concerns; however, procedures already exist to handle those concerns. Indeed, source code reviews happen all the time in the civil context, such as in patent infringement matters involving software technologies. While software makers are right to be concerned about the harm to their businesses if their code ends up in the wild, the real risk of this happening can be low if proper procedures, embodied in a suitable court-issued Protective Order, are followed by lawyers on both sides of a matter and if the court maintains oversight and demands status updates from the parties to ensure compliance and integrity in the review process. Instead of following the trial court’s approach, however, the appellate court conditional access to STRmix’s “black box” on the demonstration of specific errors in the program’s results, which seems intractable: only by looking into the black box in the first place is a party able to understand whether problems exist that affect the result. Interestingly, artificial intelligence had nothing to do with the outcome of the appellate court’s decision, yet the panel noted that “We do not underestimate the challenges facing the legal system as it confronts developments in the field of artificial intelligence.” The judges acknowledged that the notion of “machine testimony” in algorithmic decision-making matters is a subject about which there are widely divergent viewpoints in the legal community, a possible prelude to what is ahead when artificial intelligence software cases make their way through the courts in criminal or non-criminal cases.  To that, the judges cautioned, “when faced with a novel method of scientific proof, we have required a preliminary showing of general acceptance of the new technique in the relevant scientific community before the scientific evidence may be admitted at trial.” Lawyers in future artificial intelligence cases should consider how best to frame arguments concerning machine testimony in both civil and criminal contexts to improve their chances of overcoming evidentiary obstacles. Lawyers will need to effectively articulate the nature of artificial intelligence decision-making algorithms, as well as the relative roles of data scientists and model developers who make decisions about artificial intelligence model architecture, hyperparameters, data sets, model inputs, training and testing procedures, and the interpretation of results. Today’s artificial intelligence systems do not operate autonomously; there will always be humans associated with a model’s output or result and those persons may need to provide expert testimony beyond the machine’s testimony.  Even so, transparency will be important to understanding algorithmic decisions and for developing an evidentiary record in artificial intelligence cases. Read more »
  • Thanks to Bots, Transparency Emerges as Lawmakers’ Choice for Regulating Algorithmic Harm
    Digital conversational agents, like Amazon’s Alexa and Apple’s Siri, and communications agents, like those found on customer service website pages, seem to be everywhere.  The remarkable increase in the use of these and other artificial intelligence-powered “bots” in everyday customer-facing devices like smartphones, websites, desktop speakers, and toys, has been exceeded only by bots in the background that account for over half of the traffic visiting some websites.  Recently reported harms caused by certain bots have caught the attention of state and federal lawmakers.  This post briefly describes those bots and their uses and suggests reasons why new legislative efforts aimed at reducing harms caused by bad bots have so far been limited to arguably one of the least onerous tools in the lawmaker’s toolbox: transparency. Bots Explained Bots are software programmed to receive percepts from their environment, make decisions based on those percepts, and then take (preferably rational) action in their environment.  Social media bots, for example, may use machine learning algorithms to classify and “understand” incoming content, which is subsequently posted and amplified via a social media account.  Companies like Netflix uses bots on social media platforms like Facebook and Twitter to automatically communicate information about their products and services. While not all bots use machine learning and other artificial intelligence (AI) technologies, many do, such as digital conversational agents, web crawlers, and website content scrappers, the latter being programmed to “understand” content on websites using semantic natural language processing and image classifiers.  Bots that use complex human behavioral data to identify and influence or manipulate people’s attitudes or behavior (such as clicking on advertisements) often use the latest AI tech. One attribute many bots have in common is that their functionality resides in a black box.  As a result, it can be challenging (if not impossible) for an observer to explain why a bot made a particular decision or took a specific action.  While intuition can be used to infer what happens, secrets inside a black box often remain secret. Depending on their uses and characteristics, bots are often categorized by type, such as “chatbot,” which generally describes an AI technology that engages with users by replicating natural language conversations, and “helper bot,” which is sometimes used when referring to a bot that performs useful or beneficial tasks.  The term “messenger bot” may refer to a bot that communicates information, while “cyborg” is sometimes used when referring to a person who uses bot technology. Regardless of their name, complexity, or use of AI, one characteristic common to most bots is their use as agents to accomplish tasks for or on behalf of a real person.  This anonymity of agent bots makes them attractive tools for malicious purposes. Lawmakers React to Bad Bots While the spread of beneficial bots has been impressive, bots with questionable purposes have also proliferated, such as those behind disinformation campaigns used during the 2016 presidential election.  Disinformation bots, which operate social media accounts on behalf of a real person or organization, can post content to public-facing accounts.  Used extensively in marketing, these bots can receive content, either automatically or from a principal behind the scenes, related to such things as brands, campaigns, politicians, and trending topics.  When organizations create multiple accounts and use bots across those accounts to amplify each account’s content, the content can appear viral and attract attention, which may be problematic if the content is false, misleading, and biased. The success of social media bots in spreading disinformation is evident in the degree to which they have proliferated.  Twitter recently produced data showing thousands of bot-run Twitter accounts (“Twitter bots”) were created before and during the 2016 US presidential campaign by foreign actors to amplify and spread disinformation about the campaign, candidates, and related hot-button campaign issues.  Users who received content from one of these bots would have had no apparent reason to know that it came from a foreign actor. Thus, it’s easy to understand why lawmakers and stakeholders would want to target social media bots and those that use them.  In view of a recent Pew Research Center poll that found most Americans know about social media bots, and those that have heard about them overwhelmingly (80%) believe that such bots are used for malicious purposes, and with technologies to detect fake content at its source or the bias of a news source standing at only about 65-70 percent accuracy, politicians have plenty of cover to go after bots and their owners. Why Use Transparency to Address Bot Harms? The range of options for regulating disinformation bots to prevent or reduce harm could include any number of traditional legislative approaches.  These include imposing on individuals and organizations various specific criminal and civil liability standards related to the performance and uses of their technologies; establishing requirements for regular recordkeeping and reporting to authorities (which could lead to public summaries); setting thresholds for knowledge, awareness, or intent (or use of strict liability) applied to regulated activities; providing private rights of action to sue for harms caused by a regulated person’s actions, inactions, or omissions; imposing monetary remedies and incarceration for violations; and other often seen command-and-control style governance approaches.  Transparency, which is another tool lawmakers could deploy, could impose on certain regulated persons and entities that they provide information publicly or privately to an organization’s users or customers through a mechanism of notice, disclosure, and/or disclaimer (among other techniques). Transparency is a long-used principal of democratic institutions that try to balance open and accountable government action and the notion of free enterprise with the public’s right to be informed.  Examples of transparency may be found in the form of information labels on consumer products and services under consumer laws, disclosure of product endorsement interests under FTC rules, notice and disclosures in financial and real estate transactions under various related laws, employee benefits disclosures under labor and tax laws, public review disclosures in connection with laws related to government decision-making, property ownership public records disclosures under various tax and land ownership/use laws, various healthcare disclosures under state and federal health care laws, and laws covering many other areas of public life.  Of particular relevance to the disinformation problem noted above, and why transparency seems well-suited to social media bots, is current federal campaign finance laws that require those behind political ads to reveal themselves.  See 52 USC §30120 (Federal Campaign Finance Law; publication and distribution of statements and solicitations; disclaimer requirements). A recent example of a transparency rule affecting certain bot use cases is California’s bot law (SB-1001; signed by Gov. Brown on September 28, 2018).  The law, which goes into effect July 2019, will, with certain exceptions, make it unlawful for any person (including corporations or government agencies) to use a bot to communicate or interact with another person in California online with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.  A person using a bot will not be liable, however, if the person discloses using clear, conspicuous, and reasonably designed notice to inform persons with whom the bot communicates or interacts that it is a bot.  Similar federal legislation may follow, especially if legislation proposed this summer by Sen. Diane Feinstein (D-CA) and legislative proposals by Sen. Warner and others gain traction in Congress. So why would lawmakers choose transparency to regulate malicious bot technology use cases rather than use an approach that is arguably more onerous?  One possibility is that transparency is seen as minimally controversial, and therefore less likely to cause push-back by those with ties to special interests that might negatively respond to lawmakers who advocate for tougher measures.  Or, perhaps lawmakers are choosing a minimalist approach just to demonstrate that they are taking action (versus the optics associated with doing nothing).  Maybe transparency is seen as a shot across the bow warning to industry leaders: work hard to police themselves and those that use their platforms by finding technological solutions to preventing the harms caused by bots or else be subject to a harsher regulatory spotlight.  Whatever the reason(s), even something viewed as relatively easy to implement as transparency is not immune from controversy. Transparency Concerns The arguments against the use of transparency applied to bots include loss of privacy, unfairness, unnecessary disclosure, and constitutional concerns, among others. Imposing transparency requirements can potentially infringe upon First Amendment protections if drafted with one-size-fits-all applicability.  Even before California’s bots measure was signed into law, for example, critics warned of the potential chilling effect on protected speech if anonymity is lifted in the case of social media bots. Moreover, transparency may be seen as unfairly elevating the principals of openness and accountability over notions of secrecy and privacy.  Owners of agent-bots, for example, would prefer to not to give up anonymity when doing so could expose them to attacks by those with opposing viewpoints and cause more harm than the law prevents. Both concerns could be addressed by imposing transparency in a narrow set of use cases and, as in California’s bot law, using “intent to mislead” and “knowingly deceiving” thresholds for tailoring the law to specific instances of certain bad behaviors. Others might argue that transparency places too much of the burden on users to understand the information being disclosed to them and to take appropriate responsive actions.  Just ask someone who’s tried to read a financial transaction disclosure or a complex Federal Register rule-making analysis whether the transparency, openness and accountability actually made a substantive impact on their follow-up actions.  Similarly, it’s questionable whether a recipient of bot-generated content would investigate the ownership and propriety of every new posting before deciding whether to accept the content’s veracity, or whether a person engaging with an AI chatbot would forgo further engagement if he or she were informed of the artificial nature of the engagement. Conclusion The likelihood that federal transparency laws will be enacted to address the malicious use of social media bots seems low given the current political situation in the US.  And with California’s bots disclosure requirement not becoming effective until mid-2019, only time will tell whether it will succeed as a legislative tool in addressing existing bot harms or whether the delay will simply give malicious actors time to find alternative technologies to achieve their goals. Even so, transparency appears to be a leading governance approach, at least in the area of algorithmic harm, and could become a go-to approach to governing harms caused by other AI and non-AI algorithmic technologies due to its relative simplicity and ability to be narrowly tailored.  Transparency might be a suitable approach to regulating certain actions by those who publish face images using generative adversarial networks (GANs), those who create and distribute so-called “deep fake” videos, and those who provide humanistic digital communications agents, all of which involve highly-realistic content and engagements in which a user could easily be fooled into believing the content/engagement involves a person and not an artificial intelligence. Read more »
  • AI’s Problems Attract More Congressional Attention
    As contentious political issues continue to distract Congress before the November midterm elections, federal legislative proposals aimed at governing artificial intelligence (AI) have largely stalled in the Senate and House.  Since December 2017, nine AI-focused bills, such as the AI Reporting Act of 2018 (AIR Act) and the AI in Government Act of 2018, have been waiting for congressional committee attention.  Even so, there has been a noticeable uptick in the number of individual federal lawmakers looking at AI’s problems, a sign that the pendulum may be swinging in the direction favoring regulation of AI technologies. Those lawmakers taking a serious look at AI recently include Mark Warner (D-VA) and Kamala Harris (D-CA) in the Senate, and Will Hurd (R-TX) and Robin Kelly (D-IL) in the House.  Along with others in Congress, they are meeting with AI experts, issuing new policy proposals, publishing reports, and pressing federal officials for information about how government agencies are addressing AI problems, especially in hot topic areas like AI model bias, privacy, and malicious uses of AI. Sen. Warner, for example, the Senate Intelligence Committee Vice Chairman, is examining how AI technologies power disinformation.  In a draft white paper first obtained by Axios, Warner’s “Potential Policy Proposals for Regulation of Social Media and Technology Firms” raises concerns about machine learning and data collection, mentioning “deep fake” disinformation tools as one example.  Deep fakes are neural network models that can take images and video of people containing one type of content and superimpose them over different images and videos of other (or the same) people in a way that changes the original’s content and meaning.  To the viewer, the altered images and videos look like the real thing, and many who view them may be fooled into accepting the false content’s message as truth. Warner’s “suite of options” for regulating AI include one that would require platforms to provide notice when users engage with AI-based digital conversational assistants (chatbots) or visit a website the publishes content provided by content-amplification algorithms like those used during the 2016 elections.  Another Warner proposal includes modifying the Communications Decency Act’s safe harbor provisions that currently protects social media platforms who publish offending third-party content, including the aforementioned deep fakes.  This proposal would allow private rights of action against platforms who fail to take steps, after notice from victims, that prevent offending content from reappearing on their sites. Another proposal would require certain platforms to make their customer’s activity data (sufficiently anonymized) available to public interest researchers as a way to generate insight from the data that could “inform actions by regulators and Congress.”  An area of concern is the commercial use, by private tech companies, of their user’s behavior-based data (online habits) without using proper research controls.  The suggestion is that public interest researchers would evaluate a platform’s behavioral data in a way that is not driven by an underlying for-profit business model. Warner’s privacy-centered proposals include granting the Federal Trade Commission with rulemaking authority, adopting GDPR-like regulations recently implemented across the European Union states, and setting mandatory standards for algorithmic transparency (auditability and fairness). Repeating a theme in Warner’s white paper, Representatives Hurd and Kelly conclude that, even if AI technologies are immature, they have the potential to disrupt every sector of society in both anticipated and unanticipated ways.  In their “Rise of the Machines: Artificial Intelligence and its Growing Impact on U.S. Policy” report, the co-chairs of the House Oversight and Government Reform Committee make several observations and recommendations, including the need for political leadership from both Congress and the White House to achieve US global dominance in AI, the need for increased federal spending on AI research and development, means to address algorithmic accountability and transparency to remove bias in AI models, and examining whether existing regulations can address public safety and consumer risks from AI.  The challenges facing society, the lawmakers found, include the potential for job loss due to automation, privacy, model bias, and malicious use of AI technologies. Separately, Representatives Adam Schiff (D-CA), Stephanie Murphy (D-FL), and Carlos Curbelo (R-FL), in a September 13, 2018, letter to the Director of National Intelligence, are requesting the Director of National Intelligence provide Congress with a report on the spread of deep fakes (aka “hyper-realistic digital forgeries”), which they contend are allowing “malicious actors” to create depictions of individuals doing or saying things they never did, without those individuals’ consent or knowledge.  They want the intelligence agency’s report to touch on everything from assessing how foreign governments could use the technology to harm US national interests, what sort of counter-measures could be deployed to detect and deter actors from disseminating deep fakes, and if the agency needs additional legal authority to combat the problem. In a September 17, 2018, letter to the Equal Employment Opportunity Commission, Senators Harris, Patty Murray (D-WA), and Elizabeth Warren (D-MA) ask the EEOC Director to address the potentially discriminatory impacts of facial analysis technologies in the enforcement of workplace anti-discrimination laws.  As reported on this website and elsewhere, machine learning models behind facial recognition may perform poorly if they have been trained on data that is unrepresentative of data that the model sees in the wild.  For example, if training data for a facial recognition model contains primarily white male faces, the model may perform well when it sees new white male faces, but may perform poorly when it sees non-white male faces.  The Senators want to know if such technologies amplify bias in race, gender, disadvantaged, and vulnerable groups, and they have tasked the EEOC with developing guidelines for employers concerning fair uses of facial analysis technologies in the workplace. Also on September 17, 2018, Senators Harris, Richard Blumenthal (D-CT), Cory Booker (D-NJ), and Ron Wyden (D-OR), sent a similar letter to the Federal Trade Commission, expressing concerns that the bias in facial analysis technologies could be considered unfair or deceptive practices under the Federal Trade Commission Act.  Stating that “we cannot wait any longer to have a serious conversation about how we can create sound policy to address these concerns,” the Senators urge the FTC to commit to developing a set of best practices for the lawful, fair, and transparent use of facial analysis. Senators Harris and Booker, joined by Senator Cedric Richmond (D-LA), also sent a letter on September 17, 2018, to FBI Director Christopher Wray asking for the status of the FBI’s response to a 2016 General Accounting Office (GAO) comprehensive report detailing the FBI’s use of face recognition technology. The increasing attention directed toward AI by individual federal lawmakers in 2018 may merely reflect the politics of the moment rather than signal a momentum shift toward substantive federal command and control-style regulations.  But as more states join those states that have begun enacting, in the absence of federal rules, their own laws addressing AI technology use cases, federal action may inevitably follow, especially if more reports of malicious uses of AI, like election disinformation, reach more receptive ears in Congress. Read more »
  • Generative Adversarial Networks and the Rise of Fake Faces: an Intellectual Property Perspective
    The tremendous growth in the artificial intelligence (AI) sector over the last several years may be attributed in large part to the proliferation of so-called big data.  But even today, data sets of sufficient size and quality are not always available for certain applications.  That’s where a technology called generative adversarial networks (GANs) comes in.  GANs, which are neural networks comprising two separate networks (a generator and a discriminator network that face off against each another), are useful for creating new (“synthetic” or “fake”) data samples.  As a result, one of the hottest areas for AI research today involves GANs, their ever-growing use cases, and the tools to identify their fake samples in the wild.  Face image-generating GANs, in particular, have received much of the attention due to their ability to generate highly realistic faces. One of the notable features of face image-generating GANs is their ability to generate synthetic faces having particular attributes, such as desired eye and hair color, skin tone, gender, and a certain degree of “attractiveness,” among others, that by appearance are nearly indistinguishable from reality.  These fake designer face images can be combined (using feature vectors) to produce even more highly sculpted face images having custom genetic features.  A similar process using celebrity images can be used to generate fake images well-suited to targeted online or print advertisements and other purposes.  Imagine the face of someone selling you a product or service whose persona, which is customized to match your particular likes/dislikes (after all, market researchers know all about you), and which has a vague resemblance to a favorite athlete, historical figure, or celebrity.  Even though family, friends, and celebrity endorsements are seen as the best way for companies looking for high marketing conversion rates, a highly tailored GAN-generated face may one day rival those techniques. As previously discussed on this website, AI technologies involving any use of human face data, such as face detection, facial recognition, face swapping, deep fakes, and now synthetic face generation technologies, raise a number of legal (and ethical) issues.  Facial recognition (a type of regulated biometric information in some states), for example, has become a lightning rod for privacy-related laws and lawsuits.  Proponents of face image-generating GANs seem to recognize potential legal risk posed by their technology when they argue that generating synthetic faces avoids copyright restrictions (this at least implicitly acknowledges that data sets found online may contain copyrighted images scraped from the Internet).  But copyright issue may not be so clear-cut in the case of GANs.  And even if copyrights are avoided, a GAN developer may face other potential legal issues, such as those involving publicity and privacy rights. Consider the following hypothetical: GAN Developer’s face image-generating model is used to create a synthetic persona with combined features from at least two well-known public figures: Celebrity and Athlete, who own their respective publicity rights, i.e., the right to control the use of their names and likenesses, which they control through their publicity, management, legal, and/or agency teams.  Advert Co. acquires the synthetic face image sample and uses it in a national print advertising campaign that appears in leading fitness, adventure, and style magazines.  All of the real celebrity, athlete, and other images used in GAN Developer’s discriminator network are the property of Image Co.  GAN Developer did not obtain permission to use Image Co.’s images, but it also did not retain the images after its model was fully developed and used to create the synthetic face image sample. Image Co., which asserts that it owns the exclusive right to copy, reproduce, and distribute the original real images and to make derivatives thereof, sues GAN Developer and Advert Co. for copyright infringement. As a possible defense, GAN Developer might argue that its temporary use of the original copyrighted images, which were not retained after their use, was a “fair use,” and both GAN Developer and Advert Co. might further argue that the synthetic face image is an entirely new work, it is a transformative use of the original images, and it is not a derivative of the originals. With regard to their fair use argument, the Copyright Act provides a non-exhaustive list of factors to consider in deciding whether the use of a copyrighted work was an excusable fair use: “(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes; (2) the nature of the copyrighted work; (3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and (4) the effect of the use upon the potential market for or value of the copyrighted work.”  17 USC § 107.  Some of the many thoroughly-reasoned and well-cited court opinions concerning the fair use doctrine address its applicability to face images.  In just one example, a court granted summary judgment in favor of a defendant after finding that the defendant’s extracted outline features of a face from an online copyrighted photo of a mayor for use in opposition political ads was an excusable fair use.  Kienitz v. Sconnie Nation LLC, 766 F. 3d 756 (7th Cir. 2014).  Even so, no court has considered the specific fact pattern set forth in the above hypothetical involving GANs, so it remains to be seen how a court might apply the fair use doctrine in such circumstances. As for the other defenses, a derivative work is a work based on or derived from one or more already existing works.  Copyright Office Circular 14 at 1 (2013).  A derivative work incorporates some or all of a preexisting work and adds new original copyrightable authorship to that work.  A derivative works is one that generally involves transformation of the content of the preexisting work into an altered form, such as the translation of a novel into another language, the adaptation of a novel into a movie or play, the recasting of a novel as an e-book or an audiobook, or a t-shirt version of a print image.  See Authors Guild v. Google, Inc., 804 F. 3d 202, 215 (2nd Cir. 2015).  In the present hypothetical, a court might consider whether GAN Developer’s synthetic image sample is an altered form of Image Co.’s original Celebrity and Athlete images. With regard to the transformative use test, something is sufficiently transformative if it “adds something new, with a further purpose or different character, altering the first with new expression, meaning or message….” Campbell v. Acuff-Rose Music, Inc., 510 US 569, 579 (1994) (citing Leval, 103 Harv. L. Rev. at 1111). “[T]he more transformative the new work,” the more likely it may be viewed as a fair use of the original work. See id.  Thus, a court might consider whether GAN Developer’s synthetic image “is one that serves a new and different function from the original work and is not a substitute for it.”  Authors Guild, Inc. v. HathiTrust, 755 F. 3d 87, 96 (2nd Cir. 2014).  Depending on the “closeness” of the synthetic face to Celebrity’s and Athlete’s, whose features were used to design the synthetic face, a court might find that the new face is not a substitute for the originals, at least from a commercial perspective, and therefore it is sufficiently transformative.  Again, no court has considered the hypothetical GAN fact pattern, so it remains to be seen how a court might apply the transformative use test in such circumstances. Even if GAN Developer and Advert Co. successfully navigate around the copyright infringement issues, they may not be entirely out of the liability woods.  Getting back to the hypothetical, they still may face one or both of the Celebrity’s and Athlete’s misappropriation of publicity rights claims.  Publicity rights often arise in connection with the use of a person’s name or likeness for advertising purposes.  New York courts, which have a long history of dealing with publicity rights issues, have found that “a name, portrait, or picture is used ‘for advertising purposes’ if it appears in a publication which, taken in its entirety, was distributed for use in, or as part of, an advertisement or solicitation for patronage of a particular product or service.” See Scott v. WorldStarHipHop, Inc., No. 10-cv-9538 (S.D.N.Y. 2012) (citing cases). Right of publicity laws in some states cover not only a person’s persona, but extend to the unauthorized use and exploitation of that person’s voice, sound-alike voice, signature, nicknames, first name, roles or characterizations performed by that person (i.e., celebrity roles), personal catchphrases, identity, and objects closely related to or associated with the persona (i.e., celebrities associated with particular goods).  See Midler v. Ford Motor Co., 849 F.2d 460 (9th Cir. 1989) (finding advertiser liable for using sound-alike performers to approximate the vocal sound of actor Bette Midler); Waits v. Frito-Lay, Inc., 978 F.2d 1093 (9th Cir. 1992) (similar facts); Onassis v. Christian Dior, 122 Misc. 2d 603 (NY Supreme Ct. 1984) (finding advertiser liable for impermissibly misappropriating Jacqueline Kennedy Onassis’ identity for the purposes of trade and advertising where picture used to establish that identity was that of look-alike model Barbara Reynolds); White v. Samsung Electronics Am., Inc., 971 F.2d 1395 (9th Cir. 1992) (finding liability where defendant employed a robot that looked and replicated actions of Vanna White of “Wheel of Fortune” fame); Carson v. Here’s Johnny Portable Toilets, 698 F.2d 831 (6th Cir. 1983) (finding defendant liable where its advertisement associated its products with well-known “Here’s Johnny” introduction of television personality Johnny Carson); Motschenbacher v. R.J. Reynolds Tobacco Co., 498 F.2d 921 (9th Cir. 1974) (finding defendant liable where its advertisement used a distinctive phrase and race car in advertisements, and where public could unequivocally relate the phrase and the car to the famous individuals associated with the race car).  Some court’s, however, have drawn the line in the case of fictional names, even if it is closely related to a real name.  See Duncan v. Universal Music Group et al., No. 11-cv-5654 (E.D.N.Y. 2012). Thus, Advert Co. might argue that it did not misappropriate Celebrity’s and Athlete’s publicity rights for its own advantage because neither of their likenesses is generally apparent in the synthetic image.  Celebrity or Athlete might counter with evidence demonstrating the image contains the presence of sufficient genetic features, such as eye shape, that might make an observer think of them.  As some of the cases above suggest, a direct use of a name or likeness is not necessary for finding misappropriation of another’s persona. On the other hand, the burden of proof increases when identity is based on indirect means, such as through voice, association with objects, or in the case of a synthetic face, a mere resemblance. A court might also hear additional arguments against misappropriation. Similar to the transformative use test under a fair use query, Advert Co. might argue that its synthetic image adds significant creative elements such that the original images were transformed into something more than a mere likeness or imitation, or that its use of other’s likenesses was merely incidental (5 J. Thomas McCarthy, McCarthy on Trademarks and Unfair Competition § 28:7.50 (4th ed. 2014) (“The mere trivial or fleeting use of a person’s name or image in an advertisement will not trigger liability when such a usage will have only a de minimis commercial implication.”)). Other arguments that might be raised include First Amendment and perhaps a novel argument that output from a GAN model cannot constitute misappropriate because, at its core, the model simply learns for itself what features of an image’s pixel values are most useful for the purpose of characterizing images of human faces and thus neither the model nor GAN Developer had awareness of a real person’s physical features when generating a fake face.  But see In Re Facebook Biometric Information Privacy Litigation, slip op. (Dkt. 302), No. 3:15-cv-03747-JD (N.D. Cal. May 14, 2018) (finding unpersuasive a “learning” by artificial intelligence argument in the context of facial recognition) (more on this case here). This post barely touches the surface of some of the legal issues and types of evidence that might arise in a situation like the above GAN hypothetical.  One can imagine all sorts of other possible scenarios involving synthetic face images and their potential legal risks that GAN developers and others might confront. For more information about one online image data set, visit ImageNet; for an overview of GANs, see these slides (by GANs innovator Ian Goodfellow and others), this tutorial video (at 51:00 mark), and this ICLR 2018 conference paper by NVIDIA. Read more »
  • Will “Leaky” Machine Learning Usher in a New Wave of Lawsuits?
    A computer science professor at Cornell University has a new twist on Marc Andreessen’s 2011 pronouncement that software is “eating the world.”  According to Vitaly Shmatikov, it is “machine learning [that] is eating the world” today.  His personification is clear: machine learning and other applications of artificial intelligence are disrupting society at a rate that shows little sign of leveling off.  With increasing numbers of companies and individual developers producing customer-facing AI systems, it seems all but inevitable that some of those systems will create unintended and unforeseen consequences, including harm to individuals and society at large.  Researchers like Shmatikov and his colleagues are starting to reveal those consequences, including one–“leaky” machine learning models–that could have serious legal implications. In this post, the causes of action that might be asserted against a developer who publishes, either directly or via a machine learning as a service (MLaaS) cloud platform, a leaky machine learning model are explored along with possible defenses, using the lessons of cybersecurity litigation as a jumping off point. Over the last decade or more, the plaintiffs bar and the defendants bar have contributed to a body of case law now commonly referred to as cybersecurity law.  This was inevitable, given the estimated 8,000 data breaches involving 11 billion data records made public since 2005. After some well-publicized breaches, lawsuits against companies that reported data thefts began appearing more frequently on court dockets across the country.  Law firms responded by marketing “cybersecurity” practice groups whose attorneys advised clients about managing risks associated with data security and the aftermath of data exfiltrations by cybercriminals.  Today, with an estimated 70-percent of all data being generated by individuals (often related to those individuals’ activities), and with organizations globally expected to lose over 146 billion more data records between 2018 and 2023 if current cybersecurity tools are not improved (Juniper Research), the number of cybersecurity lawsuits is not expected to level off anytime soon. While data exfiltration lawsuits may be the most prevalent type of cybersecurity lawsuit today, the plaintiffs bar has begun targeting other cyber issues, such as ransomware attacks, especially those affecting healthcare facilities (in ransomware cases, malicious software freezes an organization’s computer systems until a ransom is paid; while frozen, a business may not be able to effectively deliver critical services to customers).  The same litigators who have expanding into ransomware may soon turn their attention to a new kind of cyber-like “breach”: the so-called leaky machine learning models built on thousands of personal data records. In their research, sponsored in part by the National Science Foundation (NSF) and Google, Shmatikov and his colleagues in early 2017 “uncovered multiple privacy and integrity problems in today’s [machine learning] pipelines” that could be exploited by adversaries to infer if a particular person’s data record was used to train machine learning models.  See R. Shokri, Membership Inference Attacks Against Machine Learning Models, Proceedings of the 38th IEEE Symposium on Security and Privacy (2017). They describe a health care machine learning model that could reveal to an adversary whether or not a certain patient’s data record was part of the model’s training data.  In another example, a different model trained on location and other data, used to categorize mobile users based on their movement patterns, was found to reveal by way of query whether a particular user’s location data was used. These scenarios certainly raise alarms from a privacy perspective, and one can imagine other possible instances of machine learning models revealing the kind of personal information to an attacker that might cause harm to individuals.  While actual user data may not be revealed in these attacks, the mere inference that a person’s data record was included in a data set used to train a model, what Shmatikov and previous researchers refer to as “membership inference,” could cause that person (and the thousands of others whose data records were used) embarrassment and other consequences. Assuming for the sake of argument that a membership inference disclosure of the kind described above becomes legally actionable, it is instructive to consider what businesses facing membership inference lawsuits might expect in terms of statutory and common law causes of action so they can take steps to mitigate problems and avoid contributing more cyber lawsuits to already busy court dockets (and of course avoid leaking confidential and private information).  These causes of actions could include invasion of privacy, consumer protection laws, unfair trade practices, negligence, negligent misrepresentation, innocent misrepresentation, negligent omission, breach of warranty, and emotional distress, among others.  See, e.g., Sony Gaming Networks & Cust. Data Sec. Breach Lit., 996 F.Supp. 2d 942 (S.D. Cal 2014) (evaluating data exfiltration causes of action). Negligence might be alleged, as it often is in cybersecurity cases, if plaintiff (or class action members) can establish evidence of the following four elements: the existence of a legal duty; breach of that duty; causation; and cognizable injury.  Liability might arise where defendant failed to properly safeguard and protect private personal information from unauthorized access, use, and disclosure, where such use and disclosure caused actual money or property loss or the loss of a legally-protected interest in the confidentiality and privacy of plaintiff’s/members’ personal information. Misrepresentation might be alleged if plaintiff/members can establish evidence of a misrepresentation upon which they relied and a pecuniary loss resulting from the reliance of the actionable misrepresentation. Liability under such a claim could arise if, for example, plaintiff’s data record has monetary value and a company makes representations about its use of security and data security measures in user agreements, terms of service, and/or privacy policies that turn out to be in error (for example, the company’s measures lack robustness and do not prevent an attack on a model that is found to be leaky).  In some cases, actual reliance on statements or omissions may need to be alleged. State consumer protection laws might also be alleged if plaintiff/members can establish (depending on which state law applies) deceptive misrepresentations or omissions regarding the standard, quality, or grade of a particular good or service that causes harm, such as those that mislead plaintiff/members into believing that their personal private information would be safe upon transmission to defendant when defendant knew of vulnerabilities in its data security systems. Liability could arise where defendant was deceptive in omitting notice that its machine learning model could reveal to an attacker the fact that plaintiff’s/members’ data record was used to train the model. In certain situations, plaintiff/members might have to allege with particularity the specific time, place, and content of the misrepresentation or omission if the allegations are based in fraud. For their part, defendants in membership inference cases might challenge plaintiff’s/members’ lawsuit on a number of fronts.  As an initial tactic, defendants might challenge plaintiff’s/members’ standing on the basis of failing to establish an actual injury caused by the disclosure (inference) of data record used to train a machine learning model.  See In re Science App. Intern. Corp. Backup Tape Data, 45 F. Supp. 3d 14 (D.D.C. 2014) (considering “when, exactly, the loss or theft of something as abstract as data becomes a concrete injury”). Defendants might also challenge plaintiff’s/members’ assertions that an injury is imminent or certainly impending.  In data breach cases, defendants might rely on state court decisions that denied standing where injury from a mere potential risk of future identity theft resulting from the loss of personal information was not recognized, which might also apply in a membership inference case. Defendants might also question whether permission and/or consent was given by a plaintiffs/members for the collection, storage, and use of personal data records.  This query would likely involve plaintiff’s/members’ awareness and acceptance of membership risks when they allowed their data to be used to train a machine learning model.  Defendants would likely examine whether the permission/consent given extended to and was commensurate in scope with the uses of the data records by defendant or others. Defendants might also consider applicable agreements related to a user’s data records that limited plaintiff’s/members’ choice of forum and which state laws apply, which could affect pleading and proof burdens.  Defendants might rely on language in terms of service and other agreements that provide notice of the possibility of external attacks and the risks of leaks and membership inference.  Many other challenges to a plaintiff’s/members’ allegations could also be explored. Apart from challenging causes of action on the merits, companies should also consider taking other measures like those used by companies in traditional data exfiltration cases.  These might include proactively testing their systems (in the case of machine learning models, testing for leakage) and implementing procedures to provide notice of a leaky model.  As Shmatikov and his colleagues suggest, machine learning model developers and MLaaS providers should take into account the risk that their models will leak information about their training data, warn customers about this risk, and “provide more visibility into the model and the methods that can be used to reduce this leakage.”  Machine learning companies should account for foreseeable risks and associated consequences and assess whether they are acceptable compared to the benefits received from their models. If data exfiltration, ransomware, and related cybersecurity litigation are any indication, the plaintiffs bar may one day turn its attention to the leaky machine learning problem.  If machine learning model developers and MLaaS providers want to avoid such attention and the possibility of litigation, they should not delay taking reasonable steps to mitigate the leaky machine learning model problem. Read more »
  • Trump Signs John S. McCain National Defense Authorization Act, Provides Funds for Artificial Intelligence Technologies
    By signing into law the John S. McCain National Defense Authorization Act for Fiscal Year 2019 (H.R.5515; Public Law No: 115-232; Aug. 13, 2018), the Trump Administration has established a strategy for major new national defense and national security-related initiatives involving artificial intelligence (AI) technologies.  Some of the law’s $717 billion spending authorization for fiscal year 2019 includes proposed funding to assess the current state of AI and deploy AI across the Department of Defense (DOD).  The law also recognizes that fundamental AI research is still needed within the tech-heavy military services.  The law encourages coordination between DOD activities and private industry at a time when some Silicon Valley companies are being pressured by their employees to stop engaging with DOD and other government agencies in AI. In Section 238 of the law, the Secretary of Defense is to lead “Joint Artificial Intelligence Research, Development, and Transition Activities” to include developing a set of activities within the DOD involving efforts to develop, mature, and transition AI technologies into operational use.  In Section 1051 of the law, an independent “National Security Commission on Artificial Intelligence” is to be established within the Executive Branch to review advances in AI and associated technologies, with a focus on machine learning (ML). The Commission’s mandate is to review methods and means necessary to advance the development of AI and associated technologies by the US to comprehensively address US national security and defense needs.  The Commission is to review the competitiveness of the US in AI/ML and associated technologies. “Artificial Intelligence” is defined broadly in Sec. 238 to include the following: (1) any artificial system that performs tasks under varying and unpredictable circumstances without significant human oversight, or that can learn from experience and improve performance when exposed to data sets; (2) an artificial system developed in computer software, physical hardware, or other context that solves tasks requiring human-like perception, cognition, planning, learning, communication, or physical action; (3) an artificial system designed to think or act like a human, including cognitive architectures and neural networks; (4) a set of techniques, including machine learning, that is designed to approximate a cognitive task; and (5) an artificial system designed to act rationally, including an intelligent software agent or embodied robot that achieves goals using perception, planning, reasoning, learning, communicating, decision making, and acting.  Section 1051 has a similar definition. The law does not overlook the need for governance of AI development activities, and requires regular meetings of appropriate DOD officials to integrate the functional activities of organizations and elements with respect to AI; ensure there are efficient and effective AI capabilities throughout the DOD; and develop and continuously improve research, innovation, policy, joint processes, and procedures to facilitate the development, acquisition, integration, advancement, oversight, and sustainment of AI throughout the DOD.  The DOD is also tasked with studying AI to make recommendations for legislative action relating to the technology, including recommendations to more effectively fund and organize the DOD in areas of AI. For further details, please see this earlier post. Read more »
WordPress RSS Feed Retriever by Theme Mason

Computational Intelligence

  • Full UK PhD Scholarships
    Full UK PhD scholarships in evolutionary computation/ computational intelligence/data analytics/ operations research/optimisation/simulationThanks to an arisen opportunity, we at the Operational Research (OR) group, Liverpool John Moores University (United Kingdom) may be able to offer a small number of PhD scholarships (full or tuition-fees-only depending on the quality of the candidate).There are two types of scholarships:The ones for UK/EU/settled students:Deadline 3rd March. Results to be known by end of March.provide full tuition fees for three years and,living expenses + running cost cover of about £16,500 each year (to be determined) for 3 yearsstudents have to enrol in Sept-Oct 2019Brexit will not have any impact on these scholarshipsThe ones for international students:Provide about £20,000 each year (to be determined). Students can use this amount to pay toward their tuition fees and living expenses.If the successful candidate joins one of the projects currently being run by the OR group, he/she may get additional scholarships depending on research performance and level of contribution. Regarding research topic, any area in evolutionary computation/ computational intelligence/data analytics/ operations research would be acceptable. However, I would prefer a topic that relates to one of our existing projects, which are in the following areas:OR techniques to study/mitigate the impact of climate change on transportation. For example, we have a project (with Merseyrail and Network Rail) on using data analytics and optimisation to anticipate and mitigate the impact of leaves falling on train tracks.Evolutionary computation or meta-heuristicsOR/data analytics applications in rail, in partnership with Merseyrail, Network Rail, and Rail Delivery GroupOR applications in maritime, in partnership with UK, EU and overseas portsOR applications in sustainable transportation, e.g. bicycle, e-bikes, walking, buses, emission/congestion reduction etc., in partnership with local authorities and transport authorities (e.g. the ones in Liverpool and Manchester)OR applications in logistics (e.g. bin packing, vehicle routing etc.) in partnership with logistics companies, especially those in airports, ports, and manufacturing plants (especially those in Liverpool).OR applications in manufacturing, in partnership with car manufacturers e.g. Vauxhall and Jaguar Land Rover.Interested candidates please contact Dr. Trung Thanh Nguyen with your full CV and transcripts. It is important that interested candidates contact me ASAP to ensure that we can prepare your applications in the best way to maximise your chance before the deadline of 3rd March. Read more »
  • IEEE Transactions on Neural Networks and Learning Systems, Volume 29, Issue 9, September 2018
    1. Continuous DropoutAuthor(s): Xu Shen; Xinmei Tian; Tongliang Liu; Fang Xu; Dacheng TaoPages: 3926 - 39372. Deep Manifold Learning Combined With Convolutional Neural Networks for Action RecognitionAuthor(s): Xin Chen; Jian Weng; Wei Lu; Jiaming Xu; Jiasi WengPages: 3938 - 39523. AdOn HDP-HMM: An Adaptive Online Model for Segmentation and Classification of Sequential DataAuthor(s): Ava Bargi; Richard Yi Da Xu; Massimo PiccardiPages: 3953 - 39684. Deep Learning of Constrained Autoencoders for Enhanced Understanding of DataAuthor(s): Babajide O. Ayinde; Jacek M. ZuradaPages: 3969 - 39795. Learning Methods for Dynamic Topic Modeling in Automated Behavior AnalysisAuthor(s): Olga Isupova; Danil Kuzin; Lyudmila MihaylovaPages: 3980 - 39936. Support Vector Data Descriptions and k-Means Clustering: One Class?Author(s): Nico Görnitz; Luiz Alberto Lima; Klaus-Robert Müller; Marius Kloft; Shinichi NakajimaPages: 3994 - 40067. Data-Driven Robust M-LS-SVR-Based NARX Modeling for Estimation and Control of Molten Iron Quality Indices in Blast Furnace IronmakingAuthor(s): Ping Zhou; Dongwei Guo; Hong Wang; Tianyou ChaiPages: 4007 - 40218. Detection of Sources in Non-Negative Blind Source Separation by Minimum Description Length CriterionAuthor(s): Chia-Hsiang Lin; Chong-Yung Chi; Lulu Chen; David J. Miller; Yue WangPages: 4022 - 40379. Nonparametric Coupled Bayesian Dictionary and Classifier Learning for Hyperspectral ClassificationAuthor(s): Naveed Akhtar; Ajmal MianPages: 4038 - 405010. Heterogeneous Multitask Metric Learning Across Multiple DomainsAuthor(s): Yong Luo; Yonggang Wen; Dacheng TaoPages: 4051 - 406411. Classification of Imbalanced Data by Oversampling in Kernel Space of Support Vector MachinesAuthor(s): Josey Mathew; Chee Khiang Pang; Ming Luo; Weng Hoe LeongPages: 4065 - 407612. A Novel Error-Compensation Control for a Class of High-Order Nonlinear Systems With Input DelayAuthor(s): Chao Shi; Zongcheng Liu; Xinmin Dong; Yong ChenPages: 4077 - 408713. Dimensionality Reduction in Multiple Ordinal RegressionAuthor(s): Jiabei Zeng; Yang Liu; Biao Leng; Zhang Xiong; Yiu-ming CheungPages: 4088 - 410114. A Deep Machine Learning Method for Classifying Cyclic Time Series of Biological Signals Using Time-Growing Neural NetworkAuthor(s): Arash Gharehbaghi; Maria LindénPages: 4102 - 411515. Transductive Zero-Shot Learning With Adaptive Structural EmbeddingAuthor(s): Yunlong Yu; Zhong Ji; Jichang Guo; Yanwei PangPages: 4116 - 412716. Bayesian Nonparametric Regression Modeling of Panel Data for Sequential ClassificationAuthor(s): Sihan Xiong; Yiwei Fu; Asok RayPages: 4128 - 413917. Symmetric Predictive Estimator for Biologically Plausible Neural LearningAuthor(s): David Xu; Andrew Clappison; Cameron Seth; Jeff OrchardPages: 4140 - 415118. A Distance-Based Weighted Undersampling Scheme for Support Vector Machines and its Application to Imbalanced ClassificationAuthor(s): Qi Kang; Lei Shi; MengChu Zhou; XueSong Wang; QiDi Wu; Zhi WeiPages: 4152 - 416519. Learning With Coefficient-Based Regularized Regression on Markov ResamplingAuthor(s): Luoqing Li; Weifu Li; Bin Zou; Yulong Wang; Yuan Yan Tang; Hua HanPages: 4166 - 417620. Sequential Labeling With Structural SVM Under Nondecomposable LossesAuthor(s): Guopeng Zhang; Massimo Piccardi; Ehsan Zare BorzeshiPages: 4177 - 418821. The Stability of Stochastic Coupled Systems With Time-Varying Coupling and General Topology StructureAuthor(s): Yan Liu; Wenxue Li; Jiqiang FengPages: 4189 - 420022. Stability Analysis of Quaternion-Valued Neural Networks: Decomposition and Direct ApproachesAuthor(s): Yang Liu; Dandan Zhang; Jungang Lou; Jianquan Lu; Jinde CaoPages: 4201 - 421123. On Wang k WTA With Input Noise, Output Node Stochastic, and Recurrent State NoiseAuthor(s): John Sum; Chi-Sing Leung; Kevin I.-J. HoPages: 4212 - 422224. Event-Driven Stereo Visual Tracking Algorithm to Solve Object OcclusionAuthor(s): Luis A. Camuñas-Mesa; Teresa Serrano-Gotarredona; Sio-Hoi Ieng; Ryad Benosman; Bernabé Linares-BarrancoPages: 4223 - 423725. Stability Analysis of Neural Networks With Time-Varying Delay by Constructing Novel Lyapunov FunctionalsAuthor(s): Tae H. Lee; Hieu M. Trinh; Ju H. ParkPages: 4238 - 424726. Design, Analysis, and Representation of Novel Five-Step DTZD Algorithm for Time-Varying Nonlinear OptimizationAuthor(s): Dongsheng Guo; Laicheng Yan; Zhuoyun NiePages: 4248 - 426027. Neural Observer and Adaptive Neural Control Design for a Class of Nonlinear SystemsAuthor(s): Bing Chen; Huaguang Zhang; Xiaoping Liu; Chong LinPages: 4261 - 427128. Shared Autoencoder Gaussian Process Latent Variable Model for Visual ClassificationAuthor(s): Jinxing Li; Bob Zhang; David ZhangPages: 4272 - 428629. Online Supervised Learning for Hardware-Based Multilayer Spiking Neural Networks Through the Modulation of Weight-Dependent Spike-Timing-Dependent PlasticityAuthor(s): Nan Zheng; Pinaki MazumderPages: 4287 - 430230. Neural-Network-Based Adaptive Backstepping Control With Application to Spacecraft Attitude RegulationAuthor(s): Xibin Cao; Peng Shi; Zhuoshi Li; Ming LiuPages: 4303 - 431331. Recursive Adaptive Sparse Exponential Functional Link Neural Network for Nonlinear AEC in Impulsive Noise EnvironmentAuthor(s): Sheng Zhang; Wei Xing ZhengPages: 4314 - 432332. Multilabel Prediction via Cross-View SearchAuthor(s): Xiaobo Shen; Weiwei Liu; Ivor W. Tsang; Quan-Sen Sun; Yew-Soon OngPages: 4324 - 433833. Large-Scale Metric Learning: A Voyage From Shallow to DeepAuthor(s): Masoud Faraki; Mehrtash T. Harandi; Fatih PorikliPages: 4339 - 434634. Distributed Event-Triggered Adaptive Control for Cooperative Output Regulation of Heterogeneous Multiagent Systems Under Switching TopologyAuthor(s): Ruohan Yang; Hao Zhang; Gang Feng; Huaicheng YanPages: 4347 - 435835. Event-Based Adaptive NN Tracking Control of Nonlinear Discrete-Time SystemsAuthor(s): Yuan-Xin Li; Guang-Hong YangPages: 4359 - 436936. Dynamic Analysis of Hybrid Impulsive Delayed Neural Networks With UncertaintiesAuthor(s): Bin Hu; Zhi-Hong Guan; Tong-Hui Qian; Guanrong ChenPages: 4370 - 438437. Robust Zeroing Neural-Dynamics and Its Time-Varying Disturbances Suppression Model Applied to Mobile Robot ManipulatorsAuthor(s): Dechao Chen; Yunong ZhangPages: 4385 - 439738. Multiple-Instance Ordinal RegressionAuthor(s): Yanshan Xiao; Bo Liu; Zhifeng HaoPages: 4398 - 441339. Neuroadaptive Control With Given Performance Specifications for MIMO Strict-Feedback Systems Under Nonsmooth Actuation and Output ConstraintsAuthor(s): Yongduan Song; Shuyan ZhouPages: 4414 - 442540. Lagrangean-Based Combinatorial Optimization for Large-Scale S3VMsAuthor(s): Francesco Bagattini; Paola Cappanera; Fabio SchoenPages: 4426 - 443541. Adaptive Fault-Tolerant Control for Nonlinear Systems With Multiple Sensor Faults and Unknown Control DirectionsAuthor(s): Ding Zhai; Liwei An; Xiaojian Li; Qingling ZhangPages: 4436 - 444642. Design of Distributed Observers in the Presence of Arbitrarily Large Communication DelaysAuthor(s): Kexin Liu; Jinhu Lü; Zongli LinPages: 4447 - 446143. A Solution Path Algorithm for General Parametric Quadratic Programming ProblemAuthor(s): Bin Gu; Victor S. ShengPages: 4462 - 447244. Online Density Estimation of Nonstationary Sources Using Exponential Family of DistributionsAuthor(s): Kaan Gokcesu; Suleyman S. KozatPages: 4473 - 447845. Image-Specific Classification With Local and Global DiscriminationsAuthor(s): Chunjie Zhang; Jian Cheng; Changsheng Li; Qi TianPages: 4479 - 448646. Global Asymptotic Stability for Delayed Neural Networks Using an Integral Inequality Based on Nonorthogonal PolynomialsAuthor(s): Xian-Ming Zhang; Wen-Juan Lin; Qing-Long Han; Yong He; Min WuPages: 4487 - 449347. L1-Norm Distance Minimization-Based Fast Robust Twin Support Vector k-Plane ClusteringAuthor(s): Qiaolin Ye; Henghao Zhao; Zechao Li; Xubing Yang; Shangbing Gao; Tongming Yin; Ning YePages: 4494 - 450348. Extensions to Online Feature Selection Using Bagging and BoostingAuthor(s): Gregory Ditzler; Joseph LaBarck; James Ritchie; Gail Rosen; Robi PolikarPages: 4504 - 450949. On Adaptive Boosting for System IdentificationAuthor(s): Johan Bjurgert; Patricio E. Valenzuela; Cristian R. RojasPages: 4510 - 451450. Universal Approximation by Using the Correntropy Objective FunctionAuthor(s): Mojtaba Nayyeri; Hadi Sadoghi Yazdi; Alaleh Maskooki; Modjtaba RouhaniPages: 4515 - 452151. Stability Analysis of Optimal Adaptive Control Under Value Iteration Using a Stabilizing Initial PolicyAuthor(s): Ali HeydariPages: 4522 - 452752. Object Categorization Using Class-Specific RepresentationsAuthor(s): Chunjie Zhang; Jian Cheng; Liang Li; Changsheng Li; Qi TianPages: 4528 - 453453. Improved Stability Analysis for Delayed Neural NetworksAuthor(s): Zhichen Li; Yan Bai; Congzhi Huang; Huaicheng Yan; Shicai MuPages: 4535 - 454154. Connectivity-Preserving Consensus Tracking of Uncertain Nonlinear Strict-Feedback Multiagent Systems: An Error Transformation ApproachAuthor(s): Sung Jin YooPages: 4542 - 4548 Read more »
  • Soft Computing, Volume 22, Issue 16, August 2018
    1. Optimization and decision-making with big dataAuthor(s): Xiang Li, Xiaofeng XuPages: 5197-51992. Impact of healthcare insurance on medical expense in China: new evidence from meta-analysisAuthor(s): Jian Chai, Limin Xing, Youhong Zhou, Shuo Li, K. K. Lai, Shouyang WangPages: 5201-52133. Modified bat algorithm based on covariance adaptive evolution for global optimization problemsAuthor(s): Xian Shan, Huijin ChengPages: 5215-52304. The impacts of private risk aversion magnitude and moral hazard in R&D project under uncertain environmentAuthor(s): Yiping Fu, Zhihua Chen, Yanfei LanPages: 5231-52465. Supporting consumer’s purchase decision: a method for ranking products based on online multi-attribute product ratingsAuthor(s): Zhi-Ping Fan, Yang Xi, Yang LiuPages: 5247-52616. Effect of risk attitude on outsourcing leadership preferences with demand uncertaintyAuthor(s): Huiru Chen, Yingchen Yan, Zhibing Liu, Tiantian XingPages: 5263-52787. Value-at-risk forecasts by dynamic spatial panel GJR-GARCH model for international stock indices portfolioAuthor(s): Wei-Guo Zhang, Guo-Li Mo, Fang Liu, Yong-Jun LiuPages: 5279-52978. Case-based reasoning with optimized weight derived by particle swarm optimization for software effort estimationAuthor(s): Dengsheng Wu, Jianping Li, Chunbing BaoPages: 5299-53109. Dynamic analysis for Governance–Pollution model with education promoting controlAuthor(s): Jiaorui Li, Siqi YuPages: 5311-532110. A combined neural network model for commodity price forecasting with SSAAuthor(s): Jue Wang, Xiang LiPages: 5323-533311. International investing in uncertain financial marketAuthor(s): Yi Zhang, Jinwu Gao, Qi AnPages: 5335-534612. A novel multi-attribute group decision-making method based on the MULTIMOORA with linguistic evaluationsAuthor(s): Xi Chen, Liu Zhao, Haiming LiangPages: 5347-536113. Evaluation research on commercial bank counterparty credit risk management based on new intuitionistic fuzzy methodAuthor(s): Qian Liu, Chong Wu, Lingyan LouPages: 5363-537514. A novel hybrid decision support system for thyroid disease forecastingAuthor(s): Waheed Ahmad, Ayaz Ahmad, Chuncheng Lu, Barkat Ali Khoso, Lican HuangPages: 5377-538315. A bi-level optimization model of LRP in collaborative logistics network considered backhaul no-load costAuthor(s): Xiaofeng Xu, Yao Zheng, Lean YuPages: 5385-539316. Measuring and forecasting the volatility of USD/CNY exchange rate with multi-fractal theoryAuthor(s): Limei Sun, Lina Zhu, Alec Stephenson, Jinyu WangPages: 5395-540617. Affordable levels of house prices using fuzzy linear regression analysis: the case of ShanghaiAuthor(s): Jian Zhou, Hui Zhang, Yujie Gu, Athanasios A. PantelousPages: 5407-541818. What is the value of an online retailer sharing demand forecast information?Author(s): Jinlou Zhao, Hui Zhu, Shuang ZhengPages: 5419-542819. Credibility support vector machines based on fuzzy outputsAuthor(s): Chao Wang, Xiaowei Liu, Minghu Ha, Ting ZhaoPages: 5429-543720. A novel two-sided matching decision method for technological knowledge supplier and demander considering the network collaboration effectAuthor(s): Jing Han, Bin Li, Haiming Liang, Kin Keung LaiPages: 5439-545121. Managerial compensation and research and development investment in a two-period agency settingAuthor(s): Zhiying Zhao, Guoqing Yang, Jianmin XuPages: 5453-546522. Intervention strategies for false information on two-layered networks in public crisis by numerical simulationsAuthor(s): Xiaoxia Zhu, Mengmeng LiuPages: 5467-547723. The coordination mechanisms of emergency inventory model under supply disruptionsAuthor(s): Jiaguo Liu, Huan Zhou, Junjin WangPages: 5479-548924. Evolutionary many-objective optimization based on linear assignment problem transformationsAuthor(s): Luis Miguel Antonio, José A. Molinet Berenguer, Carlos A. Coello CoelloPages: 5491-551225. Multiple-attribute decision-making method based on hesitant fuzzy linguistic Muirhead mean aggregation operatorsAuthor(s): Peide Liu, Ying Li, Maocong Zhang, Li Zhang, Juan ZhaoPages: 5513-552426. Sustainability evaluation of the supply chain with undesired outputs and dual-role factors based on double frontier network DEAAuthor(s): Yi Su, Wei SunPages: 5525-553327. Project portfolio selection based on synergy degree of composite systemAuthor(s): LiBiao Bai, Hongliang Chen, Qi Gao, Wei LuoPages: 5535-554528. Balancing strategic contributions and financial returns: a project portfolio selection model under uncertaintyAuthor(s): Yuntao Guo, Lin Wang, Suike Li, Zhi Chen, Yin ChengPages: 5547-555929. An optimal model using data envelopment analysis for uncertainty metrics in reliabilityAuthor(s): Tianpei Zu, Rui Kang, Meilin Wen, Yi YangPages: 5561-5568 Read more »
  • Soft Computing, Volume 22, Issue 15, August 2018
    Special Issue on Extensions of Fuzzy Sets in Decision Making1. A special issue on extensions of fuzzy sets in decision-makingAuthor(s): Cengiz KahramanPages: 4851-48532. Probabilistic OWA distances applied to asset managementAuthor(s): José M. Merigó, Ligang Zhou, Dejian Yu, Nabil Alrajeh, Khalid AlnowibetPages: 4855-48783. Solid waste collection system selection for smart cities based on a type-2 fuzzy multi-criteria decision techniqueAuthor(s): Murside Topaloglu, Ferhat Yarkin, Tolga KayaPages: 4879-48904. A novel interval-valued neutrosophic EDAS method: prioritization of the United Nations national sustainable development goalsAuthor(s): Ali Karaşan, Cengiz KahramanPages: 4891-49065. A new optimization meta-heuristic algorithm based on self-defense mechanism of the plants with three reproduction operatorsAuthor(s): Camilo Caraveo, Fevrier Valdez, Oscar CastilloPages: 4907-49206. Interval type-2 fuzzy c-control charts using likelihood and reduction methodsAuthor(s): Hatice Ercan-Teksen, Ahmet Sermet AnagünPages: 4921-49347. Comments on crucial and unsolved problems on Atanassov’s intuitionistic fuzzy setsAuthor(s): Piotr DworniczakPages: 4935-49398. A novel interval-valued neutrosophic AHP with cosine similarity measureAuthor(s): Eda Bolturk, Cengiz KahramanPages: 4941-49589. An advanced study on the similarity measures of intuitionistic fuzzy sets based on the set pair analysis theory and their application in decision makingAuthor(s): Harish Garg, Kamal KumarPages: 4959-497010. An interval type 2 hesitant fuzzy MCDM approach and a fuzzy c means clustering for retailer clusteringAuthor(s): Sultan Ceren Oner, Başar OztaysiPages: 4971-498711. Criteria evaluation for pricing decisions in strategic marketing management using an intuitionistic cognitive map approachAuthor(s): Elif Dogu, Y. Esra AlbayrakPages: 4989-500512. Pythagorean fuzzy engineering economic analysis of solar power plantsAuthor(s): Veysel Çoban, Sezi Çevik OnarPages: 5007-502013. Multi-objective evolutionary algorithm for tuning the Type-2 inference engine on classification taskAuthor(s): Edward C. Hinojosa, Heloisa A. CamargoPages: 5021-503114. Modeling attribute control charts by interval type-2 fuzzy setsAuthor(s): Nihal Erginel, Sevil Şentürk, Gülay YıldızPages: 5033-504115. On invariant IF-stateAuthor(s): Alžbeta Michalíková, Beloslav RiečanPages: 5043-504916. Entropy measures for Atanassov intuitionistic fuzzy sets based on divergenceAuthor(s): Ignacio Montes, Nikhil R. Pal, Susana MontesPages: 5051-507117. An enhanced fuzzy evidential DEMATEL method with its application to identify critical success factorsAuthor(s): Yuzhen Han, Yong DengPages: 5073-509018. Cloud computing technology selection based on interval-valued intuitionistic fuzzy MCDM methodsAuthor(s): Gülçin Büyüközkan, Fethullah Göçer, Orhan FeyzioğluPages: 5091-511419. Scaled aggregation operations over two- and three-dimensional index matricesAuthor(s): Velichka Traneva, Stoyan Tranev, Miroslav Stoenchev, Krassimir AtanassovPages: 5115-512020. A bipolar knowledge representation model to improve supervised fuzzy classification algorithmsAuthor(s): Guillermo Villarino, Daniel Gómez, J. Tinguaro Rodríguez, Javier MonteroPages: 5121-514621. Modeling and analysis of the simplest fuzzy PID controller of Takagi–Sugeno type with modified rule baseAuthor(s): Ritu Raj, B. M. MohanPages: 5147-516122. Expressive attribute-based keyword search with constant-size ciphertextAuthor(s): Jinguang Han, Ye Yang, Joseph K. Liu, Jiguo Li, Kaitai Liang, Jian ShenPages: 5163-517723. Task scheduling using Ant Colony Optimization in multicore architectures: a surveyAuthor(s): G. Umarani Srikanth, R. GeethaPages: 5179-5196 Read more »
  • IEEE Transactions on Neural Networks and Learning Systems, Volume 29, Issue 7, July 2018
    1. Driving Under the Influence (of Language)Author(s): Daniel Paul Barrett; Scott Alan Bronikowski; Haonan Yu; Jeffrey Mark SiskindPages: 2668 - 26832. Cascaded Subpatch Networks for Effective CNNsAuthor(s): Xiaoheng Jiang; Yanwei Pang; Manli Sun; Xuelong LiPages: 2684 - 26943. Neighborhood-Based Stopping Criterion for Contrastive DivergenceAuthor(s): Enrique Romero Merino; Ferran Mazzanti Castrillejo; Jordi Delgado PinPages: 2695 - 27044. Neural AILC for Error Tracking Against Arbitrary Initial ShiftsAuthor(s): Mingxuan Sun; Tao Wu; Lejian Chen; Guofeng ZhangPages: 2705 - 27165. RankMap: A Framework for Distributed Learning From Dense Data SetsAuthor(s): Azalia Mirhoseini; Eva L. Dyer; Ebrahim M. Songhori; Richard Baraniuk; Farinaz KoushanfarPages: 2717 - 27306. Manifold Preserving: An Intrinsic Approach for Semisupervised Distance Metric LearningAuthor(s): Shihui Ying; Zhijie Wen; Jun Shi; Yaxin Peng; Jigen Peng; Hong QiaoPages: 2731 - 27427. Transductive Regression for Data With Latent Dependence StructureAuthor(s): Nico Görnitz; Luiz Alberto Lima; Luiz Eduardo Varella; Klaus-Robert Müller; Shinichi NakajimaPages: 2743 - 27568. Variance-Constrained State Estimation for Complex Networks With Randomly Varying TopologiesAuthor(s): Hongli Dong; Nan Hou; Zidong Wang; Weijian RenPages: 2757 - 27689. Stability Analysis of Continuous-Time and Discrete-Time Quaternion-Valued Neural Networks With Linear Threshold NeuronsAuthor(s): Xiaofeng Chen; Qiankun Song; Zhongshan Li; Zhenjiang Zhao; Yurong LiuPages: 2769 - 278110. Improving Sparsity and Scalability in Regularized Nonconvex Truncated-Loss Learning ProblemsAuthor(s): Qing Tao; Gaowei Wu; Dejun ChuPages: 2782 - 279311. Policy Approximation in Policy Iteration Approximate Dynamic Programming for Discrete-Time Nonlinear SystemsAuthor(s): Wentao Guo; Jennie Si; Feng Liu; Shengwei MeiPages: 2794 - 280712. Multilateral Telecoordinated Control of Multiple Robots With Uncertain KinematicsAuthor(s): Di-Hua Zhai; Yuanqing XiaPages: 2808 - 282213. A Peak Price Tracking-Based Learning System for Portfolio SelectionAuthor(s): Zhao-Rong Lai; Dao-Qing Dai; Chuan-Xian Ren; Ke-Kun HuangPages: 2823 - 283214. Generalized Self-Organizing Maps for Automatic Determination of the Number of Clusters and Their Multiprototypes in Cluster AnalysisAuthor(s): Marian B. Gorzałczany; Filip RudzińskiPages: 2833 - 284515. Event-Triggered Distributed Approximate Optimal State and Output Control of Affine Nonlinear Interconnected SystemsAuthor(s): Vignesh Narayanan; Sarangapani JagannathanPages: 2846 - 285616. Online Feature Transformation Learning for Cross-Domain Object Category RecognitionAuthor(s): Xuesong Zhang; Yan Zhuang; Wei Wang; Witold PedryczPages: 2857 - 287117. Improving CNN Performance Accuracies With Min–Max ObjectiveAuthor(s): Weiwei Shi; Yihong Gong; Xiaoyu Tao; Jinjun Wang; Nanning ZhengPages: 2872 - 288518. Distribution-Preserving Stratified Sampling for Learning ProblemsAuthor(s): Cristiano Cervellera; Danilo MacciòPages: 2886 - 289519. Training DCNN by Combining Max-Margin, Max-Correlation Objectives, and Correntropy Loss for Multilabel Image ClassificationAuthor(s): Weiwei Shi; Yihong Gong; Xiaoyu Tao; Nanning ZhengPages: 2896 - 290820. Robust Least-Squares Support Vector Machine With Minimization of Mean and Variance of Modeling ErrorAuthor(s): Xinjiang Lu; Wenbo Liu; Chuang Zhou; Minghui HuangPages: 2909 - 292021. Joint Attributes and Event Analysis for Multimedia Event DetectionAuthor(s): Zhigang Ma; Xiaojun Chang; Zhongwen Xu; Nicu Sebe; Alexander G. HauptmannPages: 2921 - 293022. Aggregation Analysis for Competitive Multiagent Systems With Saddle Points via Switching StrategiesAuthor(s): Liying Zhu; Zhengrong XiangPages: 2931 - 294323. Learning Multimodal Parameters: A Bare-Bones Niching Differential Evolution ApproachAuthor(s): Yue-Jiao Gong; Jun Zhang; Yicong ZhouPages: 2944 - 295924. Time-Varying System Identification Using an Ultra-Orthogonal Forward Regression and Multiwavelet Basis Functions With Applications to EEGAuthor(s): Yang Li; Wei-Gang Cui; Yu-Zhu Guo; Tingwen Huang; Xiao-Feng Yang; Hua-Liang WeiPages: 2960 - 297225. Neural Decomposition of Time-Series Data for Effective GeneralizationAuthor(s): Luke B. Godfrey; Michael S. GashlerPages: 2973 - 298526. Feature Selection Based on Neighborhood Discrimination IndexAuthor(s): Changzhong Wang; Qinghua Hu; Xizhao Wang; Degang Chen; Yuhua Qian; Zhe DongPages: 2986 - 299927. Multistability of Recurrent Neural Networks With Nonmonotonic Activation Functions and Unbounded Time-Varying DelaysAuthor(s): Peng Liu; Zhigang Zeng; Jun WangPages: 3000 - 301028. Optimal Triggering of Networked Control SystemsAuthor(s): Ali HeydariPages: 3011 - 302129. Observer-Based Adaptive Fault-Tolerant Tracking Control of Nonlinear Nonstrict-Feedback SystemsAuthor(s): Chengwei Wu; Jianxing Liu; Yongyang Xiong; Ligang WuPages: 3022 - 303330. Joint Estimation of Multiple Conditional Gaussian Graphical ModelsAuthor(s): Feihu Huang; Songcan Chen; Sheng-Jun HuangPages: 3034 - 304631. Stability Analysis of Genetic Regulatory Networks With Switching Parameters and Time DelaysAuthor(s): Tingting Yu; Jianxing Liu; Yi Zeng; Xian Zhang; Qingshuang Zeng; Ligang WuPages: 3047 - 305832. Adaptive Neural Networks Prescribed Performance Control Design for Switched Interconnected Uncertain Nonlinear SystemsAuthor(s): Yongming Li; Shaocheng TongPages: 3059 - 306833. Robust and Efficient Boosting Method Using the Conditional RiskAuthor(s): Zhi Xiao; Zhe Luo; Bo Zhong; Xin DangPages: 3069 - 308334. Learning Deep Generative Models With Doubly Stochastic Gradient MCMCAuthor(s): Chao Du; Jun Zhu; Bo ZhangPages: 3084 - 309635. Discriminative Transfer Learning Using Similarities and DissimilaritiesAuthor(s): Ying Lu; Liming Chen; Alexandre Saidi; Emmanuel Dellandrea; Yunhong WangPages: 3097 - 311036. Discriminative Block-Diagonal Representation Learning for Image RecognitionAuthor(s): Zheng Zhang; Yong Xu; Ling Shao; Jian YangPages: 3111 - 312537. Robustness to Training Disturbances in SpikeProp LearningAuthor(s): Sumit Bam Shrestha; Qing SongPages: 3126 - 313938. Bayesian Neighborhood Component AnalysisAuthor(s): Dong Wang; Xiaoyang TanPages: 3140 - 315139. p th Moment Exponential Input-to-State Stability of Delayed Recurrent Neural Networks With Markovian Switching via Vector Lyapunov FunctionAuthor(s): Lei Liu; Jinde Cao; Cheng QianPages: 3152 - 316340. Distributed Adaptive Finite-Time Approach for Formation–Containment Control of Networked Nonlinear Systems Under Directed TopologyAuthor(s): Yujuan Wang; Yongduan Song; Wei RenPages: 3164 - 317541. DANoC: An Efficient Algorithm and Hardware Codesign of Deep Neural Networks on ChipAuthor(s): Xichuan Zhou; Shengli Li; Fang Tang; Shengdong Hu; Zhi Lin; Lei ZhangPages: 3176 - 318742. Causal Inference on Multidimensional Data Using Free Probability TheoryAuthor(s): Furui Liu; Lai-Wan ChanPages: 3188 - 319843. Regularized Semipaired Kernel CCA for Domain AdaptationAuthor(s): Siamak Mehrkanoon; Johan A. K. SuykensPages: 3199 - 321344. Patch Alignment Manifold MattingAuthor(s): Xuelong Li; Kang Liu; Yongsheng Dong; Dacheng TaoPages: 3214 - 322645. Supervised Learning Based on Temporal Coding in Spiking Neural NetworksAuthor(s): Hesham MostafaPages: 3227 - 323546. Multiple Structure-View Learning for Graph ClassificationAuthor(s): Jia Wu; Shirui Pan; Xingquan Zhu; Chengqi Zhang; Philip S. YuPages: 3236 - 325147. Online Heterogeneous Transfer by Hedge Ensemble of Offline and Online DecisionsAuthor(s): Yuguang Yan; Qingyao Wu; Mingkui Tan; Michael K. Ng; Huaqing Min; Ivor W. TsangPages: 3252 - 326348. Single-Input Pinning Controller Design for Reachability of Boolean NetworksAuthor(s): Fangfei Li; Huaicheng Yan; Hamid Reza KarimiPages: 3264 - 326949. Tree-Based Kernel for Graphs With Continuous AttributesAuthor(s): Giovanni Da San Martino; Nicolò Navarin; Alessandro SperdutiPages: 3270 - 327650. Sufficient Condition for the Existence of the Compact Set in the RBF Neural Network ControlAuthor(s): Jiaming Zhu; Zhiqiang Cao; Tianping Zhang; Yuequan Yang; Yang YiPages: 3277 - 328251. Delayed Feedback Control for Stabilization of Boolean Control Networks With State DelayAuthor(s): Rongjian Liu; Jianquan Lu; Yang Liu; Jinde Cao; Zheng-Guang WuPages: 3283 - 328852. Convolutional Sparse Autoencoders for Image ClassificationAuthor(s): Wei Luo; Jun Li; Jian Yang; Wei Xu; Jian ZhangPages: 3289 - 329453. An Algorithm for Finding the Most Similar Given Sized Subgraphs in Two Weighted GraphsAuthor(s): Xu Yang; Hong Qiao; Zhi-Yong LiuPages: 3295 - 330054. Normalization and Solvability of Dynamic-Algebraic Boolean NetworksAuthor(s): Yang Liu; Jinde Cao; Bowen Li; Jianquan LuPages: 3301 - 3306 Read more »
  • IEEE Transaction on Fuzzy System, Volume 26, Issue 3, June 2018
    1. Lagrange Stability for T–S Fuzzy Memristive Neural Networks with Time-Varying Delays on Time ScalesAuthor(s): Q. Xiao and Z. ZengPages: 1091-11032. A T–S Fuzzy Model Identification Approach Based on a Modified Inter Type-2 FRCM AlgorithmAuthor(s): W. Zou, C. Li and N. ZhangPages: 1104-11133. Sensor Fault Estimation of Switched Fuzzy Systems With Unknown InputAuthor(s): H. Zhang, J. Han, Y. Wang and X. LiuPages: 1114-11244. Fuzzy Remote Tracking Control for Randomly Varying Local Nonlinear Models Under Fading and Missing MeasurementsAuthor(s): J. Song, Y. Niu, J. Lam and H. K. LamPages: 1125-11375. Distributed Adaptive Fuzzy Control for Output Consensus of Heterogeneous Stochastic Nonlinear Multiagent SystemsAuthor(s): S. Li, M. J. Er and J. ZhangPages: 1138-11526. Adaptive Fuzzy Control With Prescribed Performance for Block-Triangular-Structured Nonlinear SystemsAuthor(s): Y. Li and S. TongPages: 1153-11637. Dissipativity-Based Fuzzy Integral Sliding Mode Control of Continuous-Time T-S Fuzzy SystemsAuthor(s): Y. Wang, H. Shen, H. R. Karimi and D. DuanPages: 1164-11768. A Layered-Coevolution-Based Attribute-Boosted Reduction Using Adaptive Quantum-Behavior PSO and Its Consistent Segmentation for Neonates Brain TissueAuthor(s): W. Ding, C. T. Lin, M. Prasad, Z. Cao and J. WangPages: 1177-11919. Fuzzy Model Predictive Control of Discrete-Time Systems with Time-Varying Delay and DisturbancesAuthor(s): L. Teng, Y. Wang, W. Cai and H. LiPages: 1192-120610. Finite-Time Adaptive Fuzzy Tracking Control Design for Nonlinear SystemsAuthor(s): F. Wang, B. Chen, X. Liu and C. LinPages: 1207-121611. Combination of Classifiers With Optimal Weight Based on Evidential ReasoningAuthor(s): Z. G. Liu, Q. Pan, J. Dezert and A. MartinPages: 1217-123012. Evaluating and Comparing Soft Partitions: An Approach Based on Dempster–Shafer TheoryAuthor(s): T. Denœux, S. Li and S. SriboonchittaPages: 1231-124413. Adaptive Tracking Control for a Class of Switched Nonlinear Systems Under Asynchronous SwitchingAuthor(s): D. Zhai, A. Y. Lu, J. Dong and Q. ZhangPages: 1245-125614. Incremental Perspective for Feature Selection Based on Fuzzy Rough SetsAuthor(s): Y. Yang, D. Chen, H. Wang and X. WangPages: 1257-127315. Galois Connections Between a Fuzzy Preordered Structure and a General Fuzzy StructureAuthor(s): I. P. Cabrera, P. Cordero, F. García-Pardo, M. Ojeda-Aciego and B. De BaetsPages: 1274-128716. IC-FNN: A Novel Fuzzy Neural Network With Interpretable, Intuitive, and Correlated-Contours Fuzzy Rules for Function ApproximationAuthor(s): M. M. Ebadzadeh and A. Salimi-BadrPages: 1288-130217. On Using the Shapley Value to Approximate the Choquet Integral in Cases of Uncertain ArgumentsAuthor(s): R. R. YagerPages: 1303-131018. Adaptive Fuzzy Sliding Mode Control for Network-Based Nonlinear Systems With Actuator FailuresAuthor(s): L. Chen, M. Liu, X. Huang, S. Fu and J. QiuPages: 1311-132319. Correntropy-Based Evolving Fuzzy Neural SystemAuthor(s): R. J. Bao, H. J. Rong, P. P. Angelov, B. Chen and P. K. WongPages: 1324-133820. Multiobjective Reliability Redundancy Allocation Problem With Interval Type-2 Fuzzy UncertaintyAuthor(s): P. K. Muhuri, Z. Ashraf and Q. M. D. LohaniPages: 1339-135521. Distributed Adaptive Fuzzy Control For Nonlinear Multiagent Systems Under Directed GraphsAuthor(s): C. Deng and G. H. YangPages: 1356-136622. Probability Calculation and Element Optimization of Probabilistic Hesitant Fuzzy Preference Relations Based on Expected ConsistencyAuthor(s): W. Zhou and Z. XuPages: 1367-137823. Solving High-Order Uncertain Differential Equations via Runge–Kutta MethodAuthor(s): X. Ji and J. ZhouPages: 1379-138624. On Non-commutative Residuated Lattices With Internal StatesAuthor(s): B. Zhao and P. HePages: 1387-140025. Robust ${L_1}$ Observer-Based Non-PDC Controller Design for Persistent Bounded Disturbed TS Fuzzy SystemsAuthor(s): N. Vafamand, M. H. Asemani and A. KhayatianPages: 1401-141326. Decentralized Fault Detection for Affine T–S Fuzzy Large-Scale Systems With Quantized MeasurementsAuthor(s): H. Wang and G. H. YangPages: 1414-142627. Convergence in Distribution for Uncertain Random VariablesAuthor(s): R. Gao and D. A. RalescuPages: 1427-143428. Line Integrals of Intuitionistic Fuzzy Calculus and Their PropertiesAuthor(s): Z. Ai and Z. XuPages: 1435-144629. Unknown Input-Based Observer Synthesis for a Polynomial T–S Fuzzy Model System With UncertaintiesAuthor(s): V. P. Vu, W. J. Wang, H. C. Chen and J. M. ZuradaPages: 1447-145830. Distributed Filtering for Discrete-Time T–S Fuzzy Systems With Incomplete MeasurementsAuthor(s): D. Zhang, S. K. Nguang, D. Srinivasan and L. YuPages: 1459-147131. Multi-ANFIS Model Based Synchronous Tracking Control of High-Speed Electric Multiple UnitAuthor(s): H. Yang, Y. Fu and D. WangPages: 1472-148432. A New Self-Regulated Neuro-Fuzzy Framework for Classification of EEG Signals in Motor Imagery BCIAuthor(s): A. Jafarifarmand, M. A. Badamchizadeh, S. Khanmohammadi, M. A. Nazari and B. M. TazehkandPages: 1485-149733. $Hinfty$ LMI-Based Observer Design for Nonlinear Systems via Takagi–Sugeno Models With Unmeasured Premise VariablesAuthor(s): T. M. Guerra, R. Márquez, A. Kruszewski and M. BernalPages: 1498-150934. Ensemble Fuzzy Clustering Using Cumulative Aggregation on Random ProjectionsAuthor(s): P. Rathore, J. C. Bezdek, S. M. Erfani, S. Rajasegarar and M. PalaniswamiPages: 1510-152435. Lattice-Valued Interval Operators and Its Induced Lattice-Valued Convex StructuresAuthor(s): B. Pang and Z. Y. XiuPages: 1525-153436. Deep Takagi–Sugeno–Kang Fuzzy Classifier With Shared Linguistic Fuzzy RulesAuthor(s): Y. Zhang, H. Ishibuchi and S. WangPages: 1535-154937. Stability Analysis and Control of Two-Dimensional Fuzzy Systems With Directional Time-Varying DelaysAuthor(s): L. V. Hien and H. TrinhPages: 1550-156438. A New Fuzzy Modeling Framework for Integrated Risk Prognosis and Therapy of Bladder Cancer PatientsAuthor(s): O. Obajemu, M. Mahfouf and J. W. F. CattoPages: 1565-157739. Resolution Principle in Uncertain Random EnvironmentAuthor(s): X. Yang, J. Gao and Y. NiPages: 1578-158840. Observer-Based Fuzzy Adaptive Event-Triggered Control Codesign for a Class of Uncertain Nonlinear SystemsAuthor(s): Y. X. Li and G. H. YangPages: 1589-159941. Static Output Feedback Stabilization of Positive Polynomial Fuzzy SystemsAuthor(s): A. Meng, H. K. Lam, Y. Yu, X. Li and F. LiuPages: 1600-161242. Global Asymptotic Model-Free Trajectory-Independent Tracking Control of an Uncertain Marine Vehicle: An Adaptive Universe-Based Fuzzy Control ApproachAuthor(s): N. Wang, S. F. Su, J. Yin, Z. Zheng and M. J. ErPages: 1613-162543. Information Measures in the Intuitionistic Fuzzy Framework and Their RelationshipsAuthor(s): S. Das, D. Guha and R. MesiarPages: 1626-163744. A Random Fuzzy Accelerated Degradation Model and Statistical AnalysisAuthor(s): X. Y. Li, J. P. Wu, H. G. Ma, X. Li and R. KangPages: 1638-165045. Measures of Probabilistic Interval-Valued Intuitionistic Hesitant Fuzzy Sets and the Application in Reducing Excessive Medical ExaminationsAuthor(s): Y. Zhai, Z. Xu and H. LiaoPages: 1651-167046. A Unified Collaborative Multikernel Fuzzy Clustering for Multiview DataAuthor(s): S. Zeng, X. Wang, H. Cui, C. Zheng and D. FengPages: 1671-168747. Asynchronous Piecewise Output-Feedback Control for Large-Scale Fuzzy Systems via Distributed Event-Triggering SchemesAuthor(s): Z. Zhong, Y. Zhu and H. K. LamPages: 1688-170348. Fuzzy Group Decision Making With Incomplete Information Guided by Social InfluenceAuthor(s): N. Capuano, F. Chiclana, H. Fujita, E. Herrera-Viedma and V. LoiaPages: 1704-171849. Fuzzy Bayesian LearningAuthor(s): I. Pan and D. BesterPages: 1719-173150. Observer and Adaptive Fuzzy Control Design for Nonlinear Strict-Feedback Systems With Unknown Virtual Control CoefficientsAuthor(s): B. Chen, X. Liu and C. LinPages: 1732-174351. Controllable-Domain-Based Fuzzy Rule Extraction for Copper Removal Process ControlAuthor(s): B. Zhang, C. Yang, H. Zhu, P. Shi and W. GuiPages: 1744-175652. Renewal Reward Process With Uncertain Interarrival Times and Random RewardsAuthor(s): K. Yao and J. ZhouPages: 1757-176253. Uncertainty Measures of Extended Hesitant Fuzzy Linguistic Term SetsAuthor(s): C. Wei, R. M. Rodríguez and L. MartínezPages: 1763-176854. Correction to “Detection of Resource Overload in Conditions of Project Ambiguity” [Aug 17 868-877]Author(s): M. Pelikán, H. Štiková and I. VranaPages: 1769-1769 Read more »
  • IEEE Transactions on Cognitive and Developmental Systems, Volume 10, Number 2, June 2018
    1. Guest Editorial Special Issue on Neuromorphic Computing and Cognitive SystemsH. Tang, T. Huang, J. L. Krichmar, G. Orchard and A. Basu2. Adaptive Robot Path Planning Using a Spiking Neuron Algorithm With Axonal DelaysT. Hwu, A. Y. Wang, N. Oros and J. L. Krichmar3. Neuro-Activity-Based Dynamic Path Planner for 3-D Rough TerrainA. A. Saputra, Y. Toda, J. Botzheim and N. Kubota4. EMPD: An Efficient Membrane Potential Driven Supervised Learning Algorithm for Spiking NeuronsM. Zhang, H. Qu, A. Belatreche and X. Xie5. Robotic Homunculus: Learning of Artificial Skin Representation in a Humanoid Robot Motivated by Primary Somatosensory CortexM. Hoffmann, Z. Straka, I. Farkas, M. Vavrecka and G. Metta6. A Novel Parsimonious Cause-Effect Reasoning Algorithm for Robot Imitation and Plan RecognitionG. Katz, D. W. Huang, T. Hauge, R. Gentili and J. Reggia7. Predicting Spike Trains from PMd to M1 Using Discrete Time Rescaling Targeted GLMD. Xing, C. Qian, H. Li, S. Zhang, Q. Zhang, Y. Hao, X. Zheng, Z. Wu, Y. Wang, G. Pan8. Visual Pattern Recognition Using Enhanced Visual Features and PSD-Based Learning RuleX. Xu, X. Jin, R. Yan, Q. Fang and W. Lu9. Multimodal Functional and Structural Brain Connectivity Analysis in Autism: A Preliminary Integrated Approach With EEG, fMRI, and DTIB. A. Cociu, S. Das, L. Billeci, W. Jamal, K. Maharatna, S. Calderoni, A. Narzisi, F. Muratori10. Observing and Modeling Developing Knowledge and Uncertainty During Cross-Situational Word LearningG. Kachergis and C. Yu11. Prediction Error in the PMd As a Criterion for Biological Motion Discrimination: A Computational AccountY. Kawai, Y. Nagai and M. Asada12. Learning 4-D Spatial Representations Through Perceptual Experience With HypercubesT. Miwa, Y. Sakai and S. Hashimoto13. Fuzzy Feature Extraction for Multichannel EEG ClassificationP. Y. Zhou and K. C. C. Chan14. Orthogonal Principal Coefficients Embedding for Unsupervised Subspace LearningX. Xu, S. Xiao, Z. Yi, X. Peng and Y. Liu15. A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial VehicleY. Zeng, G. Wang and B. Xu16. Biologically Inspired Self-Organizing Map Applied to Task Assignment and Path Planning of an AUV SystemD. Zhu, X. Cao, B. Sun and C. Luo17. Autonomous Discovery of Motor Constraints in an Intrinsically Motivated Vocal LearnerJ. M. Acevedo-Valle, C. Angulo and C. Moulin-Frier18. Bio-Inspired Model Learning Visual Goals and Attention Skills Through Contingencies and Intrinsic MotivationsV. Sperati and G. Baldassarre19. Seamless Integration and Coordination of Cognitive Skills in Humanoid Robots: A Deep Learning ApproachJ. Hwang and J. Tani20. Learning Temporal Intervals in Neural DynamicsB. Duran and Y. Sandamirskaya21. Quantifying Cognitive Workload in Simulated Flight Using Passive, Dry EEG MeasurementsJ. A. Blanco, M. K. Johnson, K. J. Jaquess, H. Oh, L. Lo, R. J. Gentili, B. D. Hatfield22. Enhanced Robotic Hand–Eye Coordination Inspired From Human-Like Behavioral PatternsF. Chao, Z. Zhu, C. Lin, H. Hu, L. Yang, C. Shang, C. Zhou23. Covariate Conscious Approach for Gait Recognition Based Upon Zernike Moment InvariantsH. Aggarwal and D. K. Vishwakarma24. EEG-Based Emotion Recognition Using Hierarchical Network With Subnetwork NodesY. Yang, Q. M. J. Wu, W. L. Zheng and B. L. Lu26. A Novel Biologically Inspired Visual Cognition Model: Automatic Extraction of Semantics, Formation of Integrated Concepts, and Reselection Features for AmbiguityP. Yin, H. Qiao, W. Wu, L. Qi, Y. Li, S. Zhong, B. Zhang27. Zero-Shot Image Classification Based on Deep Feature ExtractionX. Wang, C. Chen, Y. Cheng and Z. J. Wang28. A Hormone-Driven Epigenetic Mechanism for Adaptation in Autonomous RobotsJ. Lones, M. Lewis and L. Ca?amero29. Heteroscedastic Regression and Active Learning for Modeling Affordances in HumanoidsF. Stramandinoli, V. Tikhanoff, U. Pattacini and F. Nori30. Artificial Cognitive Systems That Can Answer Human Creativity Tests: An Approach and Two Case StudiesA. M. Olte?eanu, Z. Falomir and C. Freksa31. A Practical SSVEP-Based Algorithm for Perceptual Dominance Estimation in Binocular RivalryK. Tanaka, M. Tanaka, T. Kajiwara and H. O. Wang Read more »
  • Evolving Systems, Volume 9, Issue 2, June 2018, Special Section on Evolving Soft Sensors
    1. Evolving sensor systemsAuthor(s): Chrisina Jayne, Nirmalie WiratungaPages: 93-942. Predictive intelligence to the edge: impact on edge analyticsAuthor(s): Natascha Harth, Christos Anagnostopoulos, Dimitrios PezarosPages: 95-1183. Evolving ANN-based sensors for a context-aware cyber physical system of an offshore gas turbineAuthor(s): Farzan Majdani, Andrei Petrovski, Daniel DoolanPages: 119-1334. Multistatic radar classification of armed vs unarmed personnel using neural networksAuthor(s): Jarez S. Patel, Francesco Fioranelli, Matthew Ritchie, Hugh GriffithsPages: 135-1445. An evolving spatio-temporal approach for gender and age group classification with Spiking Neural NetworksAuthor(s): Fahad Bashir Alvi, Russel Pears, Nikola KasabovPages: 145-1566. Devolutionary genetic algorithms with application to the minimum labeling Steiner tree problemAuthor(s): Nassim DehouchePages: 157-1687. Modality of teaching learning based optimization algorithm to reduce the consistency ratio of the pair-wise comparison matrix in analytical hierarchy processingAuthor(s): Prashant Borkar, M. V. SarodePages: 169-180 Read more »
  • IEEE Transactions on Evolutionary Computation, Volume 22 Issue 3, June 2018
    1. Guest Editorial Special Issue on Search-Based Software EngineeringAuthor(s): Federica Sarro, Marouane Kessentini, Kalayanmoy DebPages: 3332. Constructing Cost-Aware Functional Test-Suites Using Nested Differential Evolution AlgorithmAuthor(s): Yuexing Wang, Min Zhou, Xiaoyu Song, Ming Gu, Jiaguang SunPages: 334 - 3463. Multiobjective Testing Resource Allocation Under UncertaintyAuthor(s): Roberto Pietrantuono, Pasqualina Potena, Antonio Pecchia, Daniel Rodriguez, Stefano Russo, Luis Fernández-SanzPages: 347 - 3624. Achieving Feature Location in Families of Models Through the Use of Search-Based Software EngineeringAuthor(s): Jaime Font, Lorena Arcega, Øystein Haugen, Carlos CetinaPages: 363 - 3775. Integrating Weight Assignment Strategies With NSGA-II for Supporting User Preference Multiobjective OptimizationAuthor(s): Shuai Wang, Shaukat Ali, Tao Yue, Marius LiaaenPages: 378 - 3936. An Empirical Study of Cohesion and Coupling: Balancing Optimization and DisruptionAuthor(s): Matheus Paixao, Mark Harman, Yuanyuan Zhang, Yijun YuPages: 394 - 4147. Genetic Improvement of Software: A Comprehensive SurveyAuthor(s): Justyna Petke, Saemundur O. Haraldsson, Mark Harman, William B. Langdon, David R. White, John R. WoodwardPages: 415 - 4328. Adaptively Allocating Search Effort in Challenging Many-Objective Optimization ProblemsAuthor(s): Hai-Lin Liu, Lei Chen, Qingfu Zhang, Kalyanmoy DebPages: 433 - 4489. Computing and Updating Hypervolume Contributions in Up to Four DimensionsAuthor(s): Andreia P. Guerreiro, Carlos M. FonsecaPages: 449 - 46310. Evolutionary Computation for Community Detection in Networks: A ReviewAuthor(s): Clara PizzutiPages: 464 - 48311. Escaping Local Optima Using Crossover With Emergent DiversityAuthor(s): Duc-Cuong Dang, Tobias Friedrich, Timo Kötzing, Martin S. Krejca, Per Kristian Lehre, Pietro S. Oliveto, Dirk Sudholt, Andrew M. SuttonPages: 484 - 497 Read more »
  • IEEE Transactions on Neural Networks and Learning Systems, Volume 29, Issue 6, June 2018
    1. Special Issue on Deep Reinforcement Learning and Adaptive Dynamic ProgrammingAuthor(s): Dongbin Zhao, Derong Liu, F. L. Lewis, Jose C. Principe, Stefano SquartiniPages: 2038 - 20412. Optimal and Autonomous Control Using Reinforcement Learning: A SurveyAuthor(s): Bahare Kiumarsi, Kyriakos G. Vamvoudakis, Hamidreza Modares, Frank L. LewisPages: 2042 - 20623. Applications of Deep Learning and Reinforcement Learning to Biological DataAuthor(s): Mufti Mahmud, Mohammed Shamim Kaiser, Amir Hussain, Stefano VassanelliPages: 2063 - 20794. Guided Policy Exploration for Markov Decision Processes Using an Uncertainty-Based Value-of-Information CriterionAuthor(s): Isaac J. Sledge, Matthew S. Emigh, José C. PríncipePages: 2080 - 20985. Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only StructureAuthor(s): Biao Luo, Derong Liu, Huai-Ning WuPages: 2099 - 21116. Optimal Guaranteed Cost Sliding Mode Control for Constrained-Input Nonlinear Systems With Matched and Unmatched DisturbancesAuthor(s): Huaguang Zhang, Qiuxia Qu, Geyang Xiao, Yang CuiPages: 2112 - 21267. Robust ADP Design for Continuous-Time Nonlinear Systems With Output ConstraintsAuthor(s): Bo Fan, Qinmin Yang, Xiaoyu Tang, Youxian SunPages: 2127 - 21388. Leader–Follower Output Synchronization of Linear Heterogeneous Systems With Active Leader Using Reinforcement LearningAuthor(s): Yongliang Yang, Hamidreza Modares, Donald C. Wunsch, Yixin YinPages: 2139 - 21539. Approximate Dynamic Programming: Combining Regional and Local State Following ApproximationsAuthor(s): Patryk Deptula, Joel A. Rosenfeld, Rushikesh Kamalapurkar, Warren E. DixonPages: 2154 - 216610. Suboptimal Scheduling in Switched Systems With Continuous-Time Dynamics: A Least Squares ApproachAuthor(s): Tohid Sardarmehni, Ali HeydariPages: 2167 - 217811. Optimal Fault-Tolerant Control for Discrete-Time Nonlinear Strict-Feedback Systems Based on Adaptive Critic DesignAuthor(s): Zhanshan Wang, Lei Liu, Yanming Wu, Huaguang ZhangPages: 2179 - 219112. Distributed Economic Dispatch in Microgrids Based on Cooperative Reinforcement LearningAuthor(s): Weirong Liu, Peng Zhuang, Hao Liang, Jun Peng, Zhiwu HuangPages: 2192 - 220313. Reusable Reinforcement Learning via Shallow TrailsAuthor(s): Yang Yu, Shi-Yong Chen, Qing Da, Zhi-Hua ZhouPages: 2204 - 221514. Self-Paced Prioritized Curriculum Learning With Coverage Penalty in Deep Reinforcement LearningAuthor(s): Zhipeng Ren, Daoyi Dong, Huaxiong Li, Chunlin ChenPages: 2216 - 222615. Multisource Transfer Double DQN Based on Actor LearningAuthor(s): Jie Pan, Xuesong Wang, Yuhu Cheng, Qiang YuPages: 2227 - 223816. Action-Driven Visual Object Tracking With Deep Reinforcement LearningAuthor(s): Sangdoo Yun, Jongwon Choi, Youngjoon Yoo, Kimin Yun, Jin Young ChoiPages: 2239 - 225217. Extreme Trust Region Policy Optimization for Active Object RecognitionAuthor(s): Huaping Liu, Yupei Wu, Fuchun SunPages: 2253 - 225818. Learning to Predict Consequences as a Method of Knowledge Transfer in Reinforcement LearningAuthor(s): Eric Chalmers, Edgar Bermudez Contreras, Brandon Robertson, Artur Luczak, Aaron GruberPages: 2259 - 227019. A Discrete-Time Recurrent Neural Network for Solving Rank-Deficient Matrix Equations With an Application to Output Regulation of Linear SystemsAuthor(s): Tao Liu, Jie HuangPages: 2271 - 227720. Online Learning Algorithm Based on Adaptive Control TheoryAuthor(s): Jian-Wei Liu, Jia-Jia Zhou, Mohamed S. Kamel, Xiong-Lin LuoPages: 2278 - 229321. User Preference-Based Dual-Memory Neural Model With Memory Consolidation ApproachAuthor(s): Jauwairia Nasir, Yong-Ho Yoo, Deok-Hwa Kim, Jong-Hwan KimPages: 2294 - 230822. Online HashingAuthor(s): Long-Kai Huang, Qiang Yang, Wei-Shi ZhengPages: 2309 - 232223. GoDec+: Fast and Robust Low-Rank Matrix Decomposition Based on Maximum CorrentropyAuthor(s): Kailing Guo, Liu Liu, Xiangmin Xu, Dong Xu, Dacheng TaoPages: 2323 - 233624. A Parallel Multiclassification Algorithm for Big Data Using an Extreme Learning MachineAuthor(s): Mingxing Duan, Kenli Li, Xiangke Liao, Keqin LiPages: 2337 - 235125. Nonlinear Decoupling Control With ANFIS-Based Unmodeled Dynamics Compensation for a Class of Complex Industrial ProcessesAuthor(s): Yajun Zhang, Tianyou Chai, Hong Wang, Dianhui Wang, Xinkai ChenPages: 2352 - 236626. Online Learning Algorithms Can Converge Comparably Fast as Batch LearningAuthor(s): Junhong Lin, Ding-Xuan ZhouPages: 2367 - 237827. Spiking, Bursting, and Population Dynamics in a Network of Growth Transform NeuronsAuthor(s): Ahana Gangopadhyay, Shantanu ChakrabarttyPages: 2379 - 239128. Uncertain Data Clustering in Distributed Peer-to-Peer NetworksAuthor(s): Jin Zhou, Long Chen, C. L. Philip Chen, Yingxu Wang, Han-Xiong LiPages: 2392 - 240629. Distributed Optimal Consensus Over Resource Allocation Network and Its Application to Dynamical Economic DispatchAuthor(s): Chaojie Li, Xinghuo Yu, Tingwen Huang, Xing HePages: 2407 - 241830. Distributed Adaptive Containment Control for a Class of Nonlinear Multiagent Systems With Input QuantizationAuthor(s): Chenliang Wang, Changyun Wen, Qinglei Hu, Wei Wang, Xiuyu ZhangPages: 2419 - 242831. Data-Driven Learning Control for Stochastic Nonlinear Systems: Multiple Communication Constraints and Limited StorageAuthor(s): Dong ShenPages: 2429 - 244032. Reversed Spectral HashingAuthor(s): Qingshan Liu, Guangcan Liu, Lai Li, Xiao-Tong Yuan, Meng Wang, Wei LiuPages: 2441 - 244933. Structure Learning for Deep Neural Networks Based on Multiobjective OptimizationAuthor(s): Jia Liu, Maoguo Gong, Qiguang Miao, Xiaogang Wang, Hao LiPages: 2450 - 246334. On the Dynamics of Hopfield Neural Networks on Unit QuaternionsAuthor(s): Marcos Eduardo Valle, Fidelis Zanetti de CastroPages: 2464 - 247135. End-to-End Feature-Aware Label Space Encoding for Multilabel Classification With Many ClassesAuthor(s): Zijia Lin, Guiguang Ding, Jungong Han, Ling ShaoPages: 2472 - 248736. Improved Stability and Stabilization Results for Stochastic Synchronization of Continuous-Time Semi-Markovian Jump Neural Networks With Time-Varying DelayAuthor(s): Yanling Wei, Ju H. Park, Hamid Reza Karimi, Yu-Chu Tian, Hoyoul JungPages: 2488 - 250137. Robust Latent Subspace Learning for Image ClassificationAuthor(s): Xiaozhao Fang, Shaohua Teng, Zhihui Lai, Zhaoshui He, Shengli Xie, Wai Keung WongPages: 2502 - 251538. New Splitting Criteria for Decision Trees in Stationary Data StreamsAuthor(s): Maciej Jaworski, Piotr Duda, Leszek RutkowskiPages: 2516 - 252939. A Sequential Learning Approach for Scaling Up Filter-Based Feature Subset SelectionAuthor(s): Gregory Ditzler, Robi Polikar, Gail RosenPages: 2530 - 254440. Substructural Regularization With Data-Sensitive Granularity for Sequence Transfer LearningAuthor(s): Shichang Sun, Hongbo Liu, Jiana Meng, C. L. Philip Chen, Yu YangPages: 2545 - 255741. Exponential Synchronization of Networked Chaotic Delayed Neural Network by a Hybrid Event Trigger SchemeAuthor(s): Zhongyang Fei, Chaoxu Guan, Huijun GaoPages: 2558 - 256742. Multiclass Learning With Partially Corrupted LabelsAuthor(s): Ruxin Wang, Tongliang Liu, Dacheng TaoPages: 2568 - 258043. Boundary-Eliminated Pseudoinverse Linear Discriminant for Imbalanced ProblemsAuthor(s): Yujin Zhu, Zhe Wang, Hongyuan Zha, Daqi GaoPages: 2581 - 259444. An Information-Theoretic-Cluster Visualization for Self-Organizing MapsAuthor(s): Leonardo Enzo Brito da Silva, Donald C. WunschPages: 2595 - 261345. Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear SystemsAuthor(s): Weinan Gao, Zhong-Ping JiangPages: 2614 - 262446. On the Impact of Regularization Variation on Localized Multiple Kernel LearningAuthor(s): Yina Han, Kunde Yang, Yixin Yang, Yuanliang MaPages: 2625 - 263047. Structured Learning of Tree Potentials in CRF for Image SegmentationAuthor(s): Fayao Liu, Guosheng Lin, Ruizhi Qiao, Chunhua ShenPages: 2631 - 263748. Adaptive Backstepping-Based Neural Tracking Control for MIMO Nonlinear Switched Systems Subject to Input DelaysAuthor(s): Ben Niu, Lu LiPages: 2638 - 264449. Memcomputing Numerical Inversion With Self-Organizing Logic GatesAuthor(s): Haik Manukian, Fabio L. Traversa, Massimiliano Di VentraPages: 2645 - 265050. Graph Regularized Restricted Boltzmann MachineAuthor(s): Dongdong Chen, Jiancheng Lv, Zhang YiPages: 2651 - 265951. A Self-Paced Regularization Framework for Multilabel LearningAuthor(s): Changsheng Li, Fan Wei, Junchi Yan, Xiaoyu Zhang, Qingshan Liu, Hongyuan ZhaPages: 2660 - 2666 Read more »
  • Neural Networks, Volume 104, Pages 1-124, August 2018
    1. Design of double fuzzy clustering-driven context neural networksAuthor(s): Eun-Hu Kim, Sung-Kwun Oh, Witold PedryczPages: 1-142. Bio-inspired spiking neural network for nonlinear systems controlAuthor(s): Javier Pérez, Juan A. Cabrera, Juan J. Castillo, Juan M. VelascoPages: 15-253. A frequency-domain approach to improve ANNs generalization quality via proper initializationAuthor(s): Majdi Chaari, Afef Fekih, Abdennour C. Seibi, Jalel Ben HmidaPages: 26-394. Using a model of human visual perception to improve deep learningAuthor(s): Michael Stettler, Gregory FrancisPages: 40-495. Effect of dilution in asymmetric recurrent neural networksAuthor(s): Viola Folli, Giorgio Gosti, Marco Leonetti, Giancarlo RuoccoPages: 50-59    6. Biased Dropout and Crossmap Dropout: Learning towards effective Dropout regularization in convolutional neural networkAuthor(s): Alvin Poernomo, Dae-Ki KangPages: 60-677. A deep belief network with PLSR for nonlinear system modelingAuthor(s): Junfei Qiao, Gongming Wang, Wenjing Li, Xiaoli LiPages: 68-798. Generalized pinning synchronization of delayed Cohen–Grossberg neural networks with discontinuous activationsAuthor(s): Dongshu Wang, Lihong Huang, Longkun Tang, Jinsen ZhuangPages: 80-929. Stochastic exponential synchronization of memristive neural networks with time-varying delays via quantized controlAuthor(s): Wanli Zhang, Shiju Yang, Chuandong Li, Wei Zhang, Xinsong YangPages: 93-10310. Quasi-projective synchronization of fractional-order complex-valued recurrent neural networksAuthor(s): Shuai Yang, Juan Yu, Cheng Hu, Haijun JiangPages: 104-11311. Electrical resistivity imaging inversion: An ISFLA trained kernel principal component wavelet neural network approachAuthor(s): Feibo Jiang, Li Dong, Qianwei DaiPages: 114-123 Read more »
  • Soft Computing, Volume 22, Issue 12, June 2018
    1. Solving a nonhomogeneous linear system of interval differential equationsAuthor(s): Nizami A. Gasilov, Şahin Emrah AmrahovPages: 3817-38282. N-soft sets and their decision making algorithmsAuthor(s): Fatia Fatimah, Dedi Rosadi, R. B. Fajriya Hakim, José Carlos R. AlcantudPages: 3829-38423. On the measure of M-rough approximation of L-fuzzy setsAuthor(s): Sang-Eon Han, Alexander ŠostakPages: 3843-38554. A new metaheuristic algorithm: car tracking optimization algorithmAuthor(s): Jian Chen, Hui Cai, Wei WangPages: 3857-38785. Ideals and congruences in quasi-pseudo-MV algebrasAuthor(s): Wenjuan Chen, Wieslaw A. DudekPages: 3879-38896. Solving maximal covering location problem using genetic algorithm with local refinementAuthor(s): Soumen Atta, Priya Ranjan Sinha Mahapatra, Anirban MukhopadhyayPages: 3891-39067. A novel fuzzy time series forecasting method based on the improved artificial fish swarm optimization algorithmAuthor(s): Sidong Xian, Jianfeng Zhang, Yue Xiao, Jia PangPages: 3907-39178. A novel constraint-handling technique based on dynamic weights for constrained optimization problemsAuthor(s): Chaoda Peng, Hai-Lin Liu, Fangqing GuPages: 3919-39359. Recognizing the human attention state using cardiac pulse from the noncontact and automatic-based measurementsAuthor(s): Dazhi Jiang, Bo Hu, Yifei Chen, Yu Xue, Wei Li, Zhengping LiangPages: 3937-394910. Tauberian theorems for weighted mean summability method of improper Riemann integrals of fuzzy-number-valued functionsAuthor(s): Cemal BelenPages: 3951-395711. A fuzzy decision support system for multifactor authenticationAuthor(s): Arunava Roy, Dipankar DasguptaPages: 3959-398112. A fast weak-supervised pulmonary nodule segmentation method based on modified self-adaptive FCM algorithmAuthor(s): Hui Liu, Fenghuan Geng, Qiang Guo, Caiqing Zhang, Caiming ZhangPages: 3983-399513. Adjust weight vectors in MOEA/D for bi-objective optimization problems with discontinuous Pareto frontsAuthor(s): Chunjiang Zhang, Kay Chen Tan, Loo Hay Lee, Liang GaoPages: 3997-401214. An improved method of automatic text summarization for web contents using lexical chain with semantic-related termsAuthor(s): Htet Myet Lynn, Chang Choi, Pankoo KimPages: 4013-402315. A composite particle swarm optimization approach for the composite SaaS placement in cloud environmentAuthor(s): Mohamed Amin Hajji, Haithem MezniPages: 4025-404516. Convergence analysis of standard particle swarm optimization algorithm and its improvementAuthor(s): Weiyi Qian, Ming LiPages: 4047-407017. Attribute-based fuzzy identity access control in multicloud computing environmentsAuthor(s): Wenmin Li, Qiaoyan Wen, Xuelei Li, Debiao HePages: 4071-408218. A predictive model-based image watermarking scheme using Regression Tree and Firefly algorithmAuthor(s): Behnam Kazemivash, Mohsen Ebrahimi MoghaddamPages: 4083-409819. A multiple time series-based recurrent neural network for short-term load forecastingAuthor(s): Bing Zhang, Jhen-Long Wu, Pei-Chann ChangPages: 4099-411220. Novel ranking method of interval numbers based on the Boolean matrixAuthor(s): Deqing Li, Wenyi Zeng, Qian YinPages: 4113-412221. The mean chance of ultimate ruin time in random fuzzy insurance risk modelAuthor(s): Sara Ghasemalipour, Behrouz Fathi-VajargahPages: 4123-413122. A self-adaptive and stagnation-aware breakout local search algorithm on the grid for the Steiner tree problem with revenue, budget and hop constraintsAuthor(s): Tansel Dokeroglu, Erhan MengusogluPages: 4133-415123. Valuation of European option under uncertain volatility modelAuthor(s): Sabahat Hassanzadeh, Farshid MehrdoustPages: 4153-4163 Read more »
  • Complex & Intelligent Systems All Volumes & Issues Volume 4, Issue 2, June 2018
    1. Hybrid fuzzy-based sliding-mode control approach, optimized by genetic algorithm for quadrotor unmanned aerial vehiclesAuthor(s): M. Pazooki, A. H. MazinanPages: 79-932. Forecasting of financial data: a novel fuzzy logic neural network based on error-correction concept and statisticsAuthor(s): Dusan MarcekPages: 95-1043. EFS-MI: an ensemble feature selection method for classificationAuthor(s): Nazrul Hoque, Mihir Singh, Dhruba K. BhattacharyyaPages: 105-1184. Deep neural architectures for prediction in healthcareAuthor(s): Dimitrios Kollias, Athanasios Tagaris…Pages: 119-1315. A hybrid decision support model using axiomatic fuzzy set theory in AHP and TOPSIS for multicriteria route selectionAuthor(s): Sunil Pratap Singh, Preetvanti SinghPages: 133-1436. Risk prediction in life insurance industry using supervised learning algorithmsAuthor(s): Noorhannah Boodhun, Manoj JayabalanPages: 145-154 Read more »
  • Neural Networks, Volume 103, Pages 1-150, July 2018
    1. Beyond Low-Rank Representations: Orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clusteringAuthor(s): Yang Wang, Lin WuPages: 1-82. Learning from label proportions on high-dimensional dataAuthor(s): Yong Shi, Jiabin Liu, Zhiquan Qi, Bo WangPages: 9-183. The convergence analysis of SpikeProp algorithm with smoothing regularizationAuthor(s): Junhong Zhao, Jacek M. Zurada, Jie Yang, Wei WuPages: 19-284. Multilayer bootstrap networksAuthor(s): Xiao-Lei ZhangPages: 29-435. A multivariate additive noise model for complete causal discoveryAuthor(s): Pramod Kumar Parida, Tshilidzi Marwala, Snehashish ChakravertyPages: 44-54    6. Boundedness and global robust stability analysis of delayed complex-valued neural networks with interval parameter uncertaintiesAuthor(s): Qiankun Song, Qinqin Yu, Zhenjiang Zhao, Yurong Liu, Fuad E. AlsaadiPages: 55-627. A nonnegative matrix factorization algorithm based on a discrete-time projection neural networkAuthor(s): Hangjun Che, Jun WangPages: 63-718. Personalized response generation by Dual-learning based domain adaptationAuthor(s): Min Yang, Wenting Tu, Qiang Qu, Zhou Zhao, Jia ZhuPages: 72-829. Impulsive synchronization of stochastic reaction–diffusion neural networks with mixed time delaysAuthor(s): Yin Sheng, Zhigang ZengPages: 83-9310. Information-theoretic decomposition of embodied and situated systemsAuthor(s): Federico Da RoldPages: 94-10711. The Growing Curvilinear Component Analysis (GCCA) neural networkAuthor(s): Giansalvo Cirrincione, Vincenzo Randazzo, Eros PaseroPages: 108-11712. Spiking neural networks for handwritten digit recognition—Supervised learning and network optimizationAuthor(s): Shruti R. Kulkarni, Bipin RajendranPages: 118-12713. Robust generalized Mittag-Leffler synchronization of fractional order neural networks with discontinuous activation and impulsesAuthor(s): A. Pratap, R. Raja, C. Sowmiya, O. Bagdasar, ... G. RajchakitPages: 128-14114. General memristor with applications in multilayer neural networksAuthor(s): Shiping Wen, Xudong Xie, Zheng Yan, Tingwen Huang, Zhigang ZengPages: 142-149 Read more »
  • Soft Computing, Volume 22, Issue 10, May 2018
    1. The 16th Annual UK Workshop on Computational IntelligenceAuthor(s): Plamen Angelov, Changjing Shang, Fei ChaoPages: 3123-31242. Improving fuzzy rule interpolation performance with information gain-guided antecedent weightingAuthor(s): Fangyi Li, Ying Li, Changjing Shang, Qiang ShenPages: 3125-31393. Fuzzy cerebellar model articulation controller network optimization via self-adaptive global best harmony search algorithmAuthor(s): Fei Chao, Dajun Zhou, Chih-Min Lin, Changle Zhou, Minghui Shi, Dazhen LinPages: 3141-31534. An extended Takagi–Sugeno–Kang inference system (TSK+) with fuzzy interpolation and its rule base generationAuthor(s): Jie Li, Longzhi Yang, Yanpeng Qu, Graham SextonPages: 3155-31705. An evolutionary strategy with machine learning for learning to rank in information retrievalAuthor(s): Osman Ali Sadek Ibrahim, D. Landa-SilvaPages: 3171-31856. Anomalous behaviour detection based on heterogeneous data and data fusionAuthor(s): Azliza Mohd Ali, Plamen AngelovPages: 3187-32017. Hardening against adversarial examples with the smooth gradient methodAuthor(s): Alan Mosca, George D. MagoulasPages: 3203-32138. Real-parameter unconstrained optimization based on enhanced fitness-adaptive differential evolution algorithm with novel mutationAuthor(s): Ali Wagdy Mohamed, Ponnuthurai Nagaratnam SuganthanPages: 3215-32359. An improved semantic schema modeling for genetic programmingAuthor(s): Zahra Zojaji, Mohammad Mehdi EbadzadehPages: 3237-326010. Some new solution concepts in generalized fuzzy multiobjective optimization problemsAuthor(s): Fatemeh Fayyaz Rouhbakhsh, Hassan Hassanpour, Sohrab EffatiPages: 3261-327011. A genetic algorithm-based method for optimizing the energy consumption and performance of multiprocessor systemsAuthor(s): Anju S. Pillai, Kaumudi Singh, Vijayalakshmi Saravanan, Alagan AnpalaganPages: 3271-328512. MAP approximation to the variational Bayes Gaussian mixture model and applicationAuthor(s): Kart-Leong Lim, Han WangPages: 3287-329913. Testing exponentiality for imprecise data and its applicationAuthor(s): J. Zendehdel, M. Rezaei, M. G. Akbari, R. Zarei, H. Alizadeh NoughabiPages: 3301-331214. A fuzzy K-nearest neighbor classifier to deal with imperfect dataAuthor(s): Jose M. Cadenas, M. Carmen Garrido, Raquel Martínez, Enrique MuñozPages: 3313-333015. DUK-SVD: dynamic dictionary updating for sparse representation of a long-time remote sensing image sequenceAuthor(s): Lizhe Wang, Peng Liu, Weijing Song, Kim-Kwang Raymond ChooPages: 3331-334216. Verifiable outsourcing of constrained nonlinear programming by particle swarm optimization in cloudAuthor(s): Tao Xiang, Weimin Zhang, Shigang Zhong, Jiyun YangPages: 3343-335517. Anomaly Detection in IDSs by means of unsupervised greedy learning of finite mixture modelsAuthor(s): Nicola GreggioPages: 3357-337218. An improved efficient rotation forest algorithm to predict the interactions among proteinsAuthor(s): Lei Wang, Zhu-Hong You, Shi-Xiong Xia, Xing Chen, Xin Yan, Yong ZhouPages: 3373-338119. A novel location-based encryption model using fuzzy vault schemeAuthor(s): Lin You, Yulei Chen, Bin Yan, Meng ZhanPages: 3383-339320. BFO-FMD: bacterial foraging optimization for functional module detection in protein–protein interaction networksAuthor(s): Cuicui Yang, Junzhong Ji, Aidong ZhangPages: 3395-341621. Competitive co-evolution of multi-layer perceptron classifiersAuthor(s): Marco CastellaniPages: 3417-343222. Self-adaptive differential evolution algorithm with improved mutation strategyAuthor(s): Shihao Wang, Yuzhen Li, Hongyu Yang, Hong LiuPages: 3433-344723. Reduced order modelling of linear time-invariant system using modified cuckoo search algorithmAuthor(s): Afzal Sikander, Padmanabh ThakurPages: 3449-345924. Cross-company defect prediction via semi-supervised clustering-based data filtering and MSTrA-based transfer learningAuthor(s): Xiao Yu, Man Wu, Yiheng Jian, Kwabena Ebo Bennin, Mandi Fu, Chuanxiang MaPages: 3461-3472 Read more »
  • IEEE Transactions on Neural Networks and Learning Systems, Volume 29, Issue 5, May 2018
    1. Logistic Localized Modeling of the Sample Space for Feature Selection and ClassificationAuthor(s): Narges Armanfard; James P. Reilly; Majid KomeiliPages: 1396 - 14132. Manifold Warp Segmentation of Human ActionAuthor(s): Shenglan Liu; Lin Feng; Yang Liu; Hong Qiao; Jun Wu; Wei WangPages: 1414 - 14263. Computational Model Based on Neural Network of Visual Cortex for Human Action RecognitionAuthor(s): Haihua Liu; Na Shu; Qiling Tang; Wensheng ZhangPages: 1427 - 14404. DeepX: Deep Learning Accelerator for Restricted Boltzmann Machine Artificial Neural NetworksAuthor(s): Lok-Won KimPages: 1441 - 14535. Preconditioned Stochastic Gradient DescentAuthor(s): Xi-Lin LiPages: 1454 - 14666. Global H_infty Pinning Synchronization of Complex Networks With Sampled-Data CommunicationsAuthor(s): Zhaowen Xu; Peng Shi; Hongye Su; Zheng-Guang Wu; Tingwen HuangPages: 1467 - 14767. Robust Finite-Time Stabilization of Fractional-Order Neural Networks With Discontinuous and Continuous Activation Functions Under UncertaintyAuthor(s): Zhixia Ding; Zhigang Zeng; Leimin WangPages: 1477 - 14908. Stability Analysis and Application for Delayed Neural Networks Driven by Fractional Brownian NoiseAuthor(s): Wuneng Zhou; Xianghui Zhou; Jun Yang; Jun Zhou; Dongbing TongPages: 1491 - 15029. Discriminative Sparse Neighbor Approximation for Imbalanced LearningAuthor(s): Chen Huang; Chen Change Loy; Xiaoou TangPages: 1503 - 151310. Data-Driven Multiagent Systems Consensus Tracking Using Model Free Adaptive ControlAuthor(s): Xuhui Bu; Zhongsheng Hou; Hongwei ZhangPages: 1514 - 152411. Reinforced Robust Principal Component PursuitAuthor(s): Pratik Prabhanjan Brahma; Yiyuan She; Shijie Li; Jiade Li; Dapeng WuPages: 1525 - 153812. Adaptive Boundary Iterative Learning Control for an Euler–Bernoulli Beam System With Input ConstraintAuthor(s): Wei He; Tingting Meng; Deqing Huang; Xuefang LiPages: 1539 - 154913. Synchronization of Coupled Reaction–Diffusion Neural Networks With Directed Topology via an Adaptive ApproachAuthor(s): Hao Zhang; Yin Sheng; Zhigang ZengPages: 1550 - 156114. Solving Multiextremal Problems by Using Recurrent Neural NetworksAuthor(s): Alaeddin Malek; Najmeh Hosseinipour-MahaniPages: 1562 - 157415. Multitarget Sparse Latent RegressionAuthor(s): Xiantong Zhen; Mengyang Yu; Feng Zheng; Ilanit Ben Nachum; Mousumi Bhaduri; David Laidley; Shuo LiPages: 1575 - 158616. Convolution in Convolution for Network in NetworkAuthor(s): Yanwei Pang; Manli Sun; Xiaoheng Jiang; Xuelong LiPages: 1587 - 159717. Application of LMS-Based NN Structure for Power Quality Enhancement in a Distribution Network Under Abnormal ConditionsAuthor(s): Rahul Kumar Agarwal; Ikhlaq Hussain; Bhim SinghPages: 1598 - 160718. A Confident Information First Principle for Parameter Reduction and Model Selection of Boltzmann MachinesAuthor(s): Xiaozhao Zhao; Yuexian Hou; Dawei Song; Wenjie LiPages: 1608 - 162119. AnRAD: A Neuromorphic Anomaly Detection Framework for Massive Concurrent Data StreamsAuthor(s): Qiuwen Chen; Ryan Luley; Qing Wu; Morgan Bishop; Richard W. Linderman; Qinru QiuPages: 1622 - 163620. Dynamic Uncertain Causality Graph for Knowledge Representation and Reasoning: Utilization of Statistical Data and Domain Knowledge in Complex CasesAuthor(s): Qin Zhang; Quanying YaoPages: 1637 - 165121. Recurrent Neural Networks With Auxiliary Memory UnitsAuthor(s): Jianyong Wang; Lei Zhang; Quan Guo; Zhang YiPages: 1652 - 166122. Random Forest Classifier for Zero-Shot Learning Based on Relative AttributeAuthor(s): Yuhu Cheng; Xue Qiao; Xuesong Wang; Qiang YuPages: 1662 - 167423. Improving Crowdsourced Label Quality Using Noise CorrectionAuthor(s): Jing Zhang; Victor S. Sheng; Tao Li; Xindong WuPages: 1675 - 168824. Hyperbolic Gradient Operator and Hyperbolic Back-Propagation Learning AlgorithmsAuthor(s): Tohru Nitta; Yasuaki KuroePages: 1689 - 170225. Autonomous Data Collection Using a Self-Organizing MapAuthor(s): Jan Faigl; Geoffrey A. HollingerPages: 1703 - 171526. A Preference-Based Multiobjective Evolutionary Approach for Sparse OptimizationAuthor(s): Hui Li; Qingfu Zhang; Jingda Deng; Zong-Ben XuPages: 1716 - 173127. Asynchronous State Estimation for Discrete-Time Switched Complex Networks With Communication ConstraintsAuthor(s): Dan Zhang; Qing-Guo Wang; Dipti Srinivasan; Hongyi Li; Li YuPages: 1732 - 174628. Cluster Synchronization for Interacting Clusters of Nonidentical Nodes via Intermittent Pinning ControlAuthor(s): Yu Kang; Jiahu Qin; Qichao Ma; Huijun Gao; Wei Xing ZhengPages: 1747 - 175929. SCE: A Manifold Regularized Set-Covering Method for Data PartitioningAuthor(s): Xuelong Li; Quanmao Lu; Yongsheng Dong; Dacheng TaoPages: 1760 - 177330. Efficient kNN Classification With Different Numbers of Nearest NeighborsAuthor(s): Shichao Zhang; Xuelong Li; Ming Zong; Xiaofeng Zhu; Ruili WangPages: 1774 - 178531. Manifold Regularized Correlation Object TrackingAuthor(s): Hongwei Hu; Bo Ma; Jianbing Shen; Ling ShaoPages: 1786 - 179532. Bioinspired Approach to Modeling Retinal Ganglion Cells Using System Identification TechniquesAuthor(s): Philip J. Vance; Gautham P. Das; Dermot Kerr; Sonya A. Coleman; T. Martin McGinnity; Tim Gollisch; Jian K. LiuPages: 1796 - 180833. Synchronization Criteria for Discontinuous Neural Networks With Mixed Delays via Functional Differential InclusionsAuthor(s): Dongshu Wang; Lihong Huang; Longkun TangPages: 1809 - 182134. New Conditions for Global Asymptotic Stability of Memristor Neural NetworksAuthor(s): Mauro Di Marco; Mauro Forti; Luca PancioniPages: 1822 - 183435. Doubly Nonparametric Sparse Nonnegative Matrix Factorization Based on Dependent Indian Buffet ProcessesAuthor(s): Junyu Xuan; Jie Lu; Guangquan Zhang; Richard Yi Da Xu; Xiangfeng LuoPages: 1835 - 184936. Event-Sampled Direct Adaptive NN Output- and State-Feedback Control of Uncertain Strict-Feedback SystemAuthor(s): Nathan Szanto; Vignesh Narayanan; Sarangapani JagannathanPages: 1850 - 186337. Modeling and Analysis of Beta Oscillations in the Basal GangliaAuthor(s): Chen Liu; Jiang Wang; Huiyan Li; Chris Fietkiewicz; Kenneth A. LoparoPages: 1864 - 187538. Safe Screening Rules for Accelerating Twin Support Vector Machine ClassificationAuthor(s): Xianli Pan; Zhiji Yang; Yitian Xu; Laisheng WangPages: 1876 - 188739. Dissipativity-Based Resilient Filtering of Periodic Markovian Jump Neural Networks With Quantized MeasurementsAuthor(s): Renquan Lu; Jie Tao; Peng Shi; Hongye Su; Zheng-Guang Wu; Yong XuPages: 1888 - 189940. Singularities of Three-Layered Complex-Valued Neural Networks With Split Activation FunctionAuthor(s): Masaki KobayashiPages: 1900 - 190741. A Novel Recurrent Neural Network for Manipulator Control With Improved Noise ToleranceAuthor(s): Shuai Li; Huanqing Wang; Muhammad Usman RafiquePages: 1908 - 191842. Sensitivity Analysis for Probabilistic Neural Network Structure ReductionAuthor(s): Piotr A. Kowalski; Maciej KusyPages: 1919 - 193243. Observer-Based Robust Coordinated Control of Multiagent Systems With Input SaturationAuthor(s): Xiaoling Wang; Housheng Su; Michael Z. Q. Chen; Xiaofan WangPages: 1933 - 194644. Robust Structured Nonnegative Matrix Factorization for Image RepresentationAuthor(s): Zechao Li; Jinhui Tang; Xiaofei HePages: 1947 - 196045. Extended Polynomial Growth Transforms for Design and Training of Generalized Support Vector MachinesAuthor(s): Ahana Gangopadhyay; Oindrila Chatterjee; Shantanu ChakrabarttyPages: 1961 - 197446. On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature LearningAuthor(s): Ya Li; Xinmei Tian; Tongliang Liu; Dacheng TaoPages: 1975 - 198547. Robust Multiview Data Analysis Through Collective Low-Rank SubspaceAuthor(s): Zhengming Ding; Yun FuPages: 1986 - 199748. Tensor-Factorized Neural NetworksAuthor(s): Jen-Tzung Chien; Yi-Ting BaoPages: 1998 - 201149. A Novel Pruning Algorithm for Smoothing Feedforward Neural Networks Based on Group Lasso MethodAuthor(s): Jian Wang; Chen Xu; Xifeng Yang; Jacek M. ZuradaPages: 2012 - 202450. Classification With Truncated L1 Distance KernelAuthor(s): Xiaolin Huang; Johan A. K. Suykens; Shuning Wang; Joachim Hornegger; Andreas MaierPages: 2025 - 203051. Policy Iteration Algorithm for Optimal Control of Stochastic Logical Dynamical SystemsAuthor(s) Yuhu Wu; Tielong ShenPages: 2031 - 2036 Read more »
  • IEEE Transaction on Fuzzy System, Volume 26, Issue 2, August 2018
    1. Adaptive Fuzzy Control for Pure-Feedback Nonlinear Systems With Nonaffine Functions Being Semibounded and IndifferentiableAuthor(s): Z. Liu, X. Dong, W. Xie, Y. Chen and H. LiPages: 395-4082. Uncertain Statistical Inference Models With Imprecise ObservationsAuthor(s): K. YaoPages: 409-4153. Distribution-Based Behavioral Distance for Nondeterministic Fuzzy Transition SystemsAuthor(s): H. Wu and Y. DengPages: 416-4294. The Structure and Citation Landscape of IEEE Transactions on Fuzzy Systems (1994–2015)Author(s): D. Yu, Z. Xu, Y. Kao and C. T. LinPages: 430-4425. Human Reliability Analysis Based on Human Abilities Theory ModelAuthor(s): P. A. Baziuk, J. E. N. M. Leod and S. S. RiveraPages: 443-4536. Observer-Based $H_{infty }$ Sampled-Data Fuzzy Control for a Class of Nonlinear Parabolic PDE SystemsAuthor(s): H. N. Wu and Z. P. WangPages: 454-4737. Adaptive Compensation for Infinite Number of Time-Varying Actuator Failures in Fuzzy Tracking Control of Uncertain Nonlinear SystemsAuthor(s): G. Lai, Z. Liu, C. L. P. Chen, Y. Zhang and X. ChenPages: 474-4868. Internal Fusion FunctionsAuthor(s): D. Paternain, M. J. Campión, R. Mesiar, I. Perfilieva and H. BustincePages: 487-5039. Fuzzy-Affine-Model-Based Memory Filter Design of Nonlinear Systems With Time-Varying DelayAuthor(s): Y. Wei, J. Qiu and H. R. KarimiPages: 504-51710. Linear Model With Exact Inputs and Interval-Valued Fuzzy OutputsAuthor(s): M. G. Akbari and G. HesamianPages: 518-53011. Adaptive Sliding Mode Control for Takagi–Sugeno Fuzzy Systems and Its ApplicationsAuthor(s): H. Li, J. Wang, H. Du and H. R. KarimiPages: 531-54212. New Mechanisms for Reasoning and Impacts Accumulation for Rule-Based Fuzzy Cognitive MapsAuthor(s): P. Zdanowicz and D. PetrovicPages: 543-55513. Negations With Respect to Admissible Orders in the Interval-Valued Fuzzy Set TheoryAuthor(s): M. J. Asiain, H. Bustince, R. Mesiar, A. Kolesárová and Z. TakáčPages: 556-56814. Observer-Based Adaptive Fuzzy Decentralized Optimal Control Design for Strict-Feedback Nonlinear Large-Scale SystemsAuthor(s): S. Tong, K. Sun and S. SuiPages: 569-58415. Switched Adaptive Fuzzy Tracking Control for a Class of Switched Nonlinear Systems Under Arbitrary SwitchingAuthor(s): D. Zhai, L. An, J. Dong and Q. ZhangPages: 585-59716. A Belief-Theoretical Approach to Example-Based Pose EstimationAuthor(s): W. Gong and F. CuzzolinPages: 598-61117. Fuzzy Double C-Means Clustering Based on Sparse Self-RepresentationAuthor(s): J. Gu, L. Jiao, S. Yang and F. LiuPages: 612-62618. A Measure-Theoretic Foundation for Data QualityAuthor(s): A. Bronselaer, R. De Mol and G. De TréPages: 627-63919. Cascaded Hidden Space Feature Mapping, Fuzzy Clustering, and Nonlinear Switching Regression on Large DatasetsAuthor(s): J. Wang, H. Liu, X. Qian, Y. Jiang, Z. Deng and S. WangPages: 640-65520. From Equilibrium-Based Business Intelligence to Information Conservational Quantum-Fuzzy Cryptography—A Cellular Transformation of Bipolar Fuzzy Sets to Quantum Intelligence MachineryAuthor(s): W. R. ZhangPages: 656-66921. Multiple Definite Integrals of Intuitionistic Fuzzy Calculus and Isomorphic MappingsAuthor(s): Z. Ai and Z. XuPages: 670-68022. Analysis of Data Generated From Multidimensional Type-1 and Type-2 Fuzzy Membership FunctionsAuthor(s): D. Raj, A. Gupta, B. Garg, K. Tanna and F. C. H. RheePages: 681-69323. A Fuzzy Approach for Optimal Robust Control Design of an Automotive Electronic Throttle SystemAuthor(s): H. Sun, H. Zhao, K. Huang, M. Qiu, S. Zhen and Y. H. ChenPages: 694-70424. Characterization of Uninorms With Continuous Underlying T-norm and T-conorm by Their Set of Discontinuity PointsAuthor(s): A. Mesiarová-ZemánkováPages: 705-71425. A Fast and Accurate Rule-Base Generation Method for Mamdani Fuzzy SystemsAuthor(s): L. C. Duţu, G. Mauris and P. BolonPages: 715-73326. Feature Selection With Controlled Redundancy in a Fuzzy Rule Based FrameworkAuthor(s): I. F. Chung, Y. C. Chen and N. R. PalPages: 734-74827. Distributed Saturated Control for a Class of Semilinear PDE Systems: An SOS ApproachAuthor(s): J. L. Pitarch, M. Rakhshan, M. M. Mardani and M. ShasadeghiPages: 749-76028. Unknown Input Method Based Observer Synthesis for a Discrete Time Uncertain T–S Fuzzy SystemAuthor(s): V. P. Vu, W. J. Wang, J. M. Zurada, H. C. Chen and C. H. ChiuPages: 761-77029. Spatial Filtering for EEG-Based Regression Problems in Brain–Computer Interface (BCI)Author(s): D. Wu, J. T. King, C. H. Chuang, C. T. Lin and T. P. JungPages: 771-78130. Sampled-Data Synchronization of Complex Networks With Partial Couplings and T–S Fuzzy NodesAuthor(s): Y. Wu, R. Lu, P. Shi, H. Su and Z. G. WuPages: 782-79331. Fuzzy Bag-of-Words Model for Document RepresentationAuthor(s): R. Zhao and K. MaoPages: 794-80432. Diagnostic Observer Design for T–S Fuzzy Systems: Application to Real-Time-Weighted Fault-Detection ApproachAuthor(s): L. Li, M. Chadli, S. X. Ding, J. Qiu and Y. YangPages: 805-81633. A Feature-Reduction Fuzzy Clustering Algorithm Based on Feature-Weighted EntropyAuthor(s): M. S. Yang and Y. NatalianiPages: 817-83534. Adaptive Fuzzy Decentralized Control for a Class of Strong Interconnected Nonlinear Systems With Unmodeled DynamicsAuthor(s): H. Wang, W. Liu, J. Qiu and P. X. LiuPages: 836-84635. Granular Fuzzy Regression Domain Adaptation in Takagi–Sugeno Fuzzy ModelsAuthor(s): H. Zuo, G. Zhang, W. Pedrycz, V. Behbood and J. LuPages: 847-85836. Specificity Measures and Referential SuccessAuthor(s): N. Marín, G. Rivas-Gervilla, D. Sánchez and R. R. YagerPages: 859-86837. Tracking-Error-Based Universal Adaptive Fuzzy Control for Output Tracking of Nonlinear Systems with Completely Unknown DynamicsAuthor(s): N. Wang, J. C. Sun and M. J. ErPages: 869-88338. Consensus Building for the Heterogeneous Large-Scale GDM With the Individual Concerns and SatisfactionsAuthor(s): H. Zhang, Y. Dong and E. Herrera-ViedmaPages: 884-89839. Path Tracking of an Autonomous Ground Vehicle With Different Payloads by Hierarchical Improved Fuzzy Dynamic Sliding-Mode ControlAuthor(s): C. L. Hwang, C. C. Yang and J. Y. HungPages: 899-91440. Multiobjective Programming for Type-2 Hierarchical Fuzzy Inference TreesAuthor(s): V. K. Ojha, V. Snášel and A. AbrahamPages: 915-93641. Neighbor Inconsistent Pair Selection for Attribute Reduction by Rough Set ApproachAuthor(s): J. Dai, Q. Hu, H. Hu and D. HuangPages: 937-95042. State Feedback Control for Interval Type-2 Fuzzy Systems With Time-Varying Delay and Unreliable Communication LinksAuthor(s): T. Zhao and S. DianPages: 951-96643. Modeling Self-Adaptive Software Systems by Fuzzy Rules and Petri NetsAuthor(s): Z. Ding, Y. Zhou and M. ZhouPages: 967-98444. Recurrent Mechanism and Impulse Noise Filter for Establishing ANFISAuthor(s): S. D. Nguyen, S. B. Choi and T. I. SeoPages: 985-99745. A Three-Phase Method for Group Decision Making With Interval-Valued Intuitionistic Fuzzy Preference RelationsAuthor(s): S. Wan, F. Wang and J. DongPages: 998-101046. Filtering for Fuzzy Systems With Multiplicative Sensor Noises and Multidensity QuantizerAuthor(s): Y. Xu, R. Lu, H. Peng and S. XiePages: 1011-102247. Multi-Criteria Decision Making with Interval Criteria Satisfactions Using the Golden Rule Representative ValueAuthor(s): R. R. YagerPages: 1023-103148. Inherent Fuzzy Entropy for the Improvement of EEG Complexity EvaluationAuthor(s): Z. Cao and C. T. LinPages: 1032-103549. On Nie-Tan Operator and Type-Reduction of Interval Type-2 Fuzzy SetsAuthor(s): J. Li, R. John, S. Coupland and G. KendallPages: 1036-103950. Further Results on Stabilization of Chaotic Systems Based on Fuzzy Memory Sampled-Data ControlAuthor(s): Y. Liu, J. H. Park, B. Z. Guo and Y. ShuPages: 1040-104551. Fault-Tolerant Controller Design for General Polynomial-Fuzzy-Model-Based SystemsAuthor(s): D. Ye, N. N. Diao and X. G. ZhaoPages: 1046-105152. Adaptive Output Fuzzy Fault Accommodation for a Class of Uncertain Nonlinear Systems With Multiple Time DelaysAuthor(s): L. B. Wu and G. H. YangPages: 1052-105753. Gradual Complex Numbers and Their Application for Performance Evaluation ClassifiersAuthor(s): E. L. Souza, R. H. N. Santiago, A. M. P. Canuto and R. O. NunesPages: 1058-106554. Fuzzy Tracking Control for Switched Uncertain Nonlinear Systems With Unstable Inverse DynamicsAuthor(s): G. Shao, X. Zhao, R. Wang and B. YangPages: 1066-1072 55. Ultra-Efficient Fuzzy Min/Max Circuits Based on Carbon Nanotube FETsAuthor(s): A. Bozorgmehr, M. H. Moaiyeri, K. Navi and N. BagherzadehPages: 1073-107856. A Direct Approach for Determining the Switch Points in the Karnik–Mendel AlgorithmAuthor(s): C. Chen, R. John, J. Twycross and J. M. GaribaldiPages: 1079-108557. Comments on “Fuzzy-Model-Based Quantized Guaranteed Cost Control of Nonlinear Networked SystemsAuthor(s): T. A. WeidmanPages: 1086-1088 Read more »
  • Soft Computing, Volume 22, Issue 9, May 2018
    1. Automatic clustering based on density peak detection using generalized extreme value distributionAuthor(s): Jiajun Ding, Xiongxiong He, Junqing Yuan, Bo JiangPages: 2777-27962. The logic of distributive nearlatticesAuthor(s): Luciano J. GonzálezPages: 2797-28073. Hesitant intuitionistic fuzzy entropy/cross-entropy and their applicationsAuthor(s): Dengbao Yao, Cuicui WangPages: 2809-28244. EQ-algebras with internal statesAuthor(s): Wei Wang, Xiao Long Xin, Jun Tao WangPages: 2825-28415. Nonlinear integrals and Hadamard-type inequalitiesAuthor(s): Sadegh Abbaszadeh, Ali EbadianPages: 2843-28496. Adaptive large neighborhood search heuristic for pollution-routing problem with simultaneous pickup and deliveryAuthor(s): Setareh Majidi, Seyyed-Mahdi Hosseini-Motlagh, Joshua IgnatiusPages: 2851-28657. Doubly fed induction generator (DFIG) wind turbine controlled by artificial organic networksAuthor(s): Pedro Ponce, Hiram Ponce, Arturo MolinaPages: 2867-28798. An estimation of algebraic solution for a complex interval linear systemAuthor(s): Mojtaba GhanbariPages: 2881-28909. Simultaneous selection of material and supplier under uncertainty in carton box industries: a fuzzy possibilistic multi-criteria approachAuthor(s): Sam Mosallaeipour, Ali Mahmoodirad, Sadegh Niroomand, Bela VizvariPages: 2891-290510. Discovering taxonomies in Wikipedia by means of grammatical evolutionAuthor(s): Lourdes Araujo, Juan Martinez-Romo, Andrés DuquePages: 2907-291911. Optimal solution to orbital three-player defense problems using impulsive transferAuthor(s): Yifang Liu, Renfu Li, Lin Hu, Zhao-quan CaiPages: 2921-293412. A self-adaptive artificial bee colony algorithm based on global best for global optimizationAuthor(s): Yu Xue, Jiongming Jiang, Binping Zhao, Tinghuai MaPages: 2935-295213. Design of digital IIR filter with low quantization error using hybrid optimization techniqueAuthor(s): N. Agrawal, A. Kumar, Varun BajajPages: 2953-297114. Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learningAuthor(s): Hui-Chuan Chu, William Wei-Jen Tsai, Min-Ju Liao, Yuh-Min ChenPages: 2973-299915. Characterizations of ordered semihypergroups by the properties of their intersectional-soft generalized bi-hyperidealsAuthor(s): Muhammad Farooq, Asghar Khan, Bijan DavvazPages: 3001-301016. Byzantine-resilient dual gossip membership management in cloudsAuthor(s): JongBeom Lim, Kwang-Sik Chung, HwaMin Lee, Kangbin Yim, Heonchang YuPages: 3011-302217. Modeling of Curie temperature of manganite for magnetic refrigeration application using manual search and hybrid gravitational-based support vector regressionAuthor(s): Taoreed O. Owolabi, Kabiru O. Akande, Sunday O. OlatunjiPages: 3023-303218. Engineering simulated evolution for integrated power optimization in data centersAuthor(s): Sadiq M. Sait, Ali RazaPages: 3033-304819. A novel method for Bayesian networks structure learning based on Breeding Swarm algorithmAuthor(s): Ali Reza Khanteymoori, Mohammad-H. Olyaee, Omid Abbaszadeh, Maryam ValianPages: 3049-306020. Implementation of scalable fuzzy relational operations in MapReduceAuthor(s): Elham S. Khorasani, Matthew Cremeens, Zhenge ZhaoPages: 3061-307521. A new attitude coupled with fuzzy thinking for solving fuzzy equationsAuthor(s): T. Allahviranloo, I. Perfilieva, F. AbbasiPages: 3077-309522. Return scaling cross-correlation forecasting by stochastic time strength neural network in financial market dynamicsAuthor(s): Haiyan Mo, Jun WangPages: 3097-310923. Improving file locality in multi-keyword top-k search based on clusteringAuthor(s): Lanxiang Chen, Nan Zhang, Kuan-Ching Li, Shuibing He, Linbing QiuPages: 3111-3121 Read more »
  • IEEE Transactions on Neural Networks and Learning Systems, Volume 29, Issue 4, April 2018
    1. Multicolumn RBF NetworkAuthor(s): Ammar O. Hoori; Yuichi MotaiPages: 766 - 7782. A Regularized SNPOM for Stable Parameter Estimation of RBF-AR(X) ModelAuthor(s): Xiaoyong Zeng; Hui Peng; Feng ZhouPages: 779 - 7913. A Fast Algorithm of Convex Hull Vertices Selection for Online ClassificationAuthor(s): Shuguang Ding; Xiangli Nie; Hong Qiao; Bo ZhangPages: 792 - 8064. Adaptive Antisynchronization of Multilayer Reaction–Diffusion Neural NetworksAuthor(s): Yanzhi Wu; Lu Liu; Jiangping Hu; Gang FengPages: 807 - 8185. Synchronization for the Realization-Dependent Probabilistic Boolean NetworksAuthor(s): Hongwei Chen; Jinling Liang; Jianquan Lu; Jianlong QiuPages: 819 - 8316. Adaptive Tracking Control for Robots With an Interneural Computing SchemeAuthor(s): Feng-Sheng Tsai; Sheng-Yi Hsu; Mau-Hsiang ShihPages: 832 - 8447. Robust Estimation for Neural Networks With Randomly Occurring Distributed Delays and Markovian Jump CouplingAuthor(s): Yong Xu; Renquan Lu; Peng Shi; Jie Tao; Shengli XiePages: 845 - 8558. Finite-Time Stabilization of Delayed Memristive Neural Networks: Discontinuous State-Feedback and Adaptive Control ApproachAuthor(s): Zuowei Cai; Lihong HuangPages: 856 - 8689. Identifying a Probabilistic Boolean Threshold Network From SamplesAuthor(s): Avraham A. Melkman; Xiaoqing Cheng; Wai-Ki Ching; Tatsuya AkutsuPages: 869 - 88110. Online Nonlinear AUC Maximization for Imbalanced Data SetsAuthor(s): Junjie Hu; Haiqin Yang; Michael R. Lyu; Irwin King; Anthony Man-Cho SoPages: 882 - 89511. Feature Combination via ClusteringAuthor(s): Jian Hou; Huijun Gao; Xuelong LiPages: 896 - 90712. Impulsive Effects on Quasi-Synchronization of Neural Networks With Parameter Mismatches and Time-Varying DelayAuthor(s): Ze Tang; Ju H. Park; Jianwen FengPages: 908 - 91913. Multivariate Time-Series Classification Using the Hidden-Unit Logistic ModelAuthor(s): Wenjie Pei; Hamdi Dibeklioğlu; David M. J. Tax; Laurens van der MaatenPages: 920 - 93114. Manifold Regularized Reinforcement LearningAuthor(s): Hongliang Li; Derong Liu; Ding WangPages: 932 - 94315. Adaptive Unsupervised Feature Selection With Structure RegularizationAuthor(s): Minnan Luo; Feiping Nie; Xiaojun Chang; Yi Yang; Alexander G. Hauptmann; Qinghua ZhengPages: 944 - 95616. Adaptive Dynamic Programming for Discrete-Time Zero-Sum GamesAuthor(s): Qinglai Wei; Derong Liu; Qiao Lin; Ruizhuo SongPages: 957 - 96917. Quantization-Based Adaptive Actor-Critic Tracking Control With Tracking Error ConstraintsAuthor(s): Quan-Yong Fan; Guang-Hong Yang; Dan YePages: 970 - 98018. A Collaborative Neurodynamic Approach to Multiple-Objective Distributed OptimizationAuthor(s): Shaofu Yang; Qingshan Liu; Jun WangPages: 981 - 99219. On Mixed Data and Event Driven Design for Adaptive-Critic-Based Nonlinear H_infty ControlAuthor(s): Ding Wang; Chaoxu Mu; Derong Liu; Hongwen MaPages: 993 - 100520. Regularized Label Relaxation Linear RegressionAuthor(s): Xiaozhao Fang; Yong Xu; Xuelong Li; Zhihui Lai; Wai Keung Wong; Bingwu FangPages: 1006 - 101821. Collaborative Random Faces-Guided Encoders for Pose-Invariant Face Representation LearningAuthor(s): Ming Shao; Yizhe Zhang; Yun FuPages: 1019 - 103222. Model-Based Adaptive Event-Triggered Control of Strict-Feedback Nonlinear SystemsAuthor(s): Yuan-Xin Li; Guang-Hong YangPages: 1033 - 104523. Finite-Time State Estimation for Recurrent Delayed Neural Networks With Component-Based Event-Triggering ProtocolAuthor(s): Licheng Wang; Zidong Wang; Guoliang Wei; Fuad E. AlsaadiPages: 1046 - 105724. Quantum-Inspired Multidirectional Associative Memory With a Self-Convergent Iterative LearningAuthor(s): Naoki Masuyama; Chu Kiong Loo; Manjeevan Seera; Naoyuki KubotaPages: 1058 - 106825. Safe Exploration Algorithms for Reinforcement Learning ControllersAuthor(s): Tommaso Mannucci; Erik-Jan van Kampen; Cornelis de Visser; Qiping ChuPages: 1069 - 108126. Robustness Analysis on Dual Neural Network-based k WTA With Input NoiseAuthor(s): Ruibin Feng; Chi-Sing Leung; John SumPages: 1082 - 109427. Differentiation-Free Multiswitching Neuroadaptive Control of Strict-Feedback SystemsAuthor(s): Jeng-Tze Huang; Thanh-Phong PhamPages: 1095 - 110728. Pattern-Based NN Control of a Class of Uncertain Nonlinear SystemsAuthor(s): Feifei Yang; Cong WangPages: 1108 - 111929. Fast-Solving Quasi-Optimal LS-S3VM Based on an Extended Candidate SetAuthor(s): Yuefeng Ma; Xun Liang; James T. Kwok; Jianping Li; Xiaoping Zhou; Haiyan ZhangPages: 1120 - 113130. Homotopy Methods Based on L0-Norm for Compressed SensingAuthor(s): Zhengshan Dong; Wenxing ZhuPages: 1132 - 114631. Adaptive Neural Output Feedback Control for Nonstrict-Feedback Stochastic Nonlinear Systems With Unknown Backlash-Like Hysteresis and Unknown Control DirectionsAuthor(s): Zhaoxu Yu; Shugang Li; Zhaosheng Yu; Fangfei LiPages: 1147 - 116032. Probabilistic Distance for Mixtures of Independent Component AnalyzersAuthor(s): Gonzalo Safont; Addisson Salazar; Luis Vergara; Enriqueta Gómez; Vicente VillanuevaPages: 1161 - 117333. Adaptive Fuzzy Neural Network Control for a Constrained Robot Using Impedance LearningAuthor(s): Wei He; Yiting DongPages: 1174 - 118634. Simultaneous Bayesian Clustering and Feature Selection Through Student’s t Mixtures ModelAuthor(s): Jianyong Sun; Aimin Zhou; Simeon Keates; Shengbin LiaoPages: 1187 - 119935. Finite-Horizon H_infty Tracking Control for Unknown Nonlinear Systems With Saturating ActuatorsAuthor(s): Huaguang Zhang; Xiaohong Cui; Yanhong Luo; He JiangPages: 1200 - 121236. Boundary Control of Linear Uncertain 1-D Parabolic PDE Using Approximate Dynamic ProgrammingAuthor(s): Behzad Talaei; Sarangapani Jagannathan; John SinglerPages: 1213 - 122537. Discrete-Time Stable Generalized Self-Learning Optimal Control With Approximation ErrorsAuthor(s): Qinglai Wei; Benkai Li; Ruizhuo SongPages: 1226 - 123838. Neural-Network-Based Robust Optimal Tracking Control for MIMO Discrete-Time Systems With Unknown Uncertainty Using Adaptive Critic DesignAuthor(s): Lei Liu; Zhanshan Wang; Huaguang ZhangPages: 1239 - 125139. Partition-Based Solutions of Static Logical Networks With ApplicationsAuthor(s): Yupeng Qiao; Hongsheng Qi; Daizhan ChengPages: 1252 - 126240. Output Feedback-Based Boundary Control of Uncertain Coupled Semilinear Parabolic PDE Using Neurodynamic ProgrammingAuthor(s): Behzad Talaei; Sarangapani Jagannathan; John SinglerPages: 1263 - 127441. Adaptive Sliding Mode Control of Dynamic Systems Using Double Loop Recurrent Neural Network StructureAuthor(s): Juntao Fei; Cheng LuPages: 1275 - 128642. SPANNER: A Self-Repairing Spiking Neural Network Hardware ArchitectureAuthor(s): Junxiu Liu; Jim Harkin; Liam P. Maguire; Liam J. McDaid; John J. WadePages: 1287 - 130043. Learning a No-Reference Quality Assessment Model of Enhanced Images With Big DataAuthor(s): Ke Gu; Dacheng Tao; Jun-Fei Qiao; Weisi LinPages: 1301 - 131344. Person Reidentification Based on Elastic ProjectionsAuthor(s): Xuelong Li; Lina Liu; Xiaoqiang LuPages: 1314 - 132745. k-Times Markov Sampling for SVMCAuthor(s): Bin Zou; Chen Xu; Yang Lu; Yuan Yan Tang; Jie Xu; Xinge YouPages: 1328 - 134146. Neural Network Learning and Robust Stabilization of Nonlinear Systems With Dynamic UncertaintiesAuthor(s): Ding Wang; Derong Liu; Chaoxu Mu; Yun ZhangPages: 1342 - 135147. Groupwise Retargeted Least-Squares RegressionAuthor(s): Lingfeng Wang; Chunhong PanPages: 1352 - 135848. Consensus of Hybrid Multi-Agent SystemsAuthor(s): Yuanshi Zheng; Jingying Ma; Long WangPages: 1359 - 136549. Decomposition of Rotor Hopfield Neural Networks Using Complex NumbersAuthor(s): Masaki KobayashiPages: 1366 - 137050. A Stochastic Spiking Neural Network for Virtual ScreeningAuthor(s): A. Morro; V. Canals; A. Oliver; M. L. Alomar; F. Galán-Prado; P. J. Ballester; J. L. RossellóPages: 1371 - 137551. State Estimation for Static Neural Networks With Time-Varying Delays Based on an Improved Reciprocally Convex InequalityAuthor(s): Xian-Ming Zhang; Qing-Long HanPages: 1376 - 138152. Improving KPCA Online Extraction by Orthonormalization in the Feature SpaceAuthor(s): João B. O. Souza Filho; Paulo S. R. DinizPages: 1382 - 138753. Observability of Automata Networks: Fixed and Switching CasesAuthor(s): Rui Li; Yiguang Hong; Xingyuan WangPages: 1388 - 1394 Read more »
  • Soft Computing, Volume 22, Issue 8, April 2018
    1. An orthogonal parallel symbiotic organism search algorithm embodied with augmented Lagrange multiplier for solving constrained optimization problemsAuthor(s): Arnapurna Panda, Sabyasachi PaniPages: 2429-24472. A hybrid recommender system for e-learning based on context awareness and sequential pattern miningAuthor(s): John K. Tarus, Zhendong Niu, Dorothy KaluiPages: 2449-24613. Multiple attribute similarity hypermatchingAuthor(s): Ronald Yager, Fred Petry, Paul ElmorePages: 2463-24694. On central limit theorems for IV-eventsAuthor(s): Piotr Nowak, Olgierd HryniewiczPages: 2471-24835. Sum of observables on MV-effect algebrasAuthor(s): Anatolij DvurečenskijPages: 2485-24936. A novel three-party password-based authenticated key exchange protocol with user anonymity based on chaotic mapsAuthor(s): Chun-Ta Li, Chin-Ling Chen, Cheng-Chi Lee, Chi-Yao Weng, Chien-Ming ChenPages: 2495-25067. SAR image edge detection via sparse representationAuthor(s): Xiaole Ma, Shuaiqi Liu, Shaohai Hu, Peng Geng, Ming Liu, Jie ZhaoPages: 2507-25158. A privacy-preserving fuzzy interest matching protocol for friends finding in social networksAuthor(s): Xu An Wang, Fatos Xhafa, Xiaoshuang Luo, Shuaiwei Zhang, Yong DingPages: 2517-25269. The example application of genetic algorithm for the framework of cultural and creative brand design in Tamsui Historical MuseumAuthor(s): Shang-Chia Chiou, Yun-Ciao WangPages: 2527-254510. A GA-based simulation system for WMNs: comparison analysis for different number of flows, client distributions, DCF and EDCA functionsAuthor(s): Admir Barolli, Tetsuya Oda, Keita Matsuo, Miralda Cuka, Leonard Barolli…Pages: 2547-255511. On possibility-degree formulae for ranking interval numbersAuthor(s): Fang Liu, Li-Hua Pan, Zu-Lin Liu, Ya-Nan PengPages: 2557-256512. Multidimensional knapsack problem optimization using a binary particle swarm model with genetic operationsAuthor(s): Luis Fernando Mingo López, Nuria Gómez Blas, Alberto Arteta AlbertPages: 2567-258213. A novel subgraph K+-isomorphism method in social network based on graph similarity detectionAuthor(s): Huan Rong, Tinghuai Ma, Meili Tang, Jie CaoPages: 2583-260114. A sophisticated PSO based on multi-level adaptation and purposeful detectionAuthor(s): Xuewen Xia, Bojian Wang, Chengwang Xie, Zhongbo Hu, Bo Wei, Chang JinPages: 2603-261815. Sensor fusion of camera, GPS and IMU using fuzzy adaptive multiple motion modelsAuthor(s): Erkan Bostanci, Betul Bostanci, Nadia Kanwal, Adrian F. ClarkPages: 2619-263216. A new hybrid intuitionistic approach for new product selectionAuthor(s): Kumru Didem Atalay, Gülin Feryal CanPages: 2633-264017. Particle state change algorithmAuthor(s): Xiang Feng, Hanyu Xu, Huiqun Yu, Fei LuoPages: 2641-266618. Optimization of neural network for nonlinear discrete time system using modified quaternion firefly algorithm: case study of Indian currency exchange rate predictionAuthor(s): Uday Pratap Singh, Sanjeev JainPages: 2667-268119. B-spline collocation and self-adapting differential evolution (jDE) algorithm for a singularly perturbed convection–diffusion problemAuthor(s): Xu-Qiong Luo, Li-Bin Liu, Aijia Ouyang, Guangqing LongPages: 2683-269320. Detecting and quantifying ambiguity: a neural network approachAuthor(s): Rui Ligeiro, R. Vilela MendesPages: 2695-270321. Decentralized adaptive optimal stabilization of nonlinear systems with matched interconnectionsAuthor(s): Chaoxu Mu, Changyin Sun, Ding Wang, Aiguo Song, Chengshan QianPages: 2705-271522. Multi-functional nearest-neighbour classificationAuthor(s): Yanpeng Qu, Changjing Shang, Neil Mac Parthaláin, Wei Wu, Qiang ShenPages: 2717-273023. Adaptive contents for interactive TV guided by machine learning based on predictive sentiment analysis of dataAuthor(s): Victor M. Mondragon, Vicente García-Díaz, Carlos Porcel…Pages: 2731-275224. Improved fuzzy C-means clustering in the process of exudates detection using mathematical morphologyAuthor(s): Kittipol Wisaeng, Worawat Sa-ngiamviboolPages: 2753-276425. The rainbow spanning forest problemAuthor(s): Francesco Carrabs, Carmine Cerrone, Raffaele Cerulli, Selene SilvestriPages: 2765-2776 Read more »
  • Neural Networks, Volume 100, Pages 1-94, April 2018
    1. Coupled generative adversarial stacked Auto-encoder: CoGASAAuthor(s): Mohammad Ahangar Kiasari, Dennis Singh Moirangthem, Minho LeePages: 1-9    2. -synchronization and Mittag-Leffler synchronization for the fractional-order memristive neural networks with delays and discontinuous neuron activationsAuthor(s): Jiejie Chen, Boshan Chen, Zhigang ZengPages: 10-24    3. Merging weighted SVMs for parallel incremental learningAuthor(s): Lei Zhu, Kazushi Ikeda, Shaoning Pang, Tao Ban, Abdolhossein SarrafzadehPages: 25-384. Accelerated low-rank representation for subspace clustering and semi-supervised classification on large-scale dataAuthor(s): Jicong Fan, Zhaoyang Tian, Mingbo Zhao, Tommy W.S. ChowPages: 39-485. GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization frameworkAuthor(s): Lei Deng, Peng Jiao, Jing Pei, Zhenzhi Wu, Guoqi LiPages: 49-586. Distant supervision for neural relation extraction integrated with word attention and property featuresAuthor(s): Jianfeng Qu, Dantong Ouyang, Wen Hua, Yuxin Ye, Ximing LiPages: 59-697. Extracting cardiac dynamics within ECG signal for human identification and cardiovascular diseases classificationAuthor(s): Muqing Deng, Cong Wang, Min Tang, Tongjia ZhengPages: 70-838. Max-plus and min-plus projection autoassociative morphological memories and their compositions for pattern classificationAuthor(s): Alex Santana dos Santos, Marcos Eduardo VallePages: 84-94 Read more »
  • Soft Computing, Volume 22, Issue 5, March 2018
    Special issue on Real Time Image Processing Systems using Fuzzy and Rough Sets Techniques1. Real-time image processing systems using fuzzy and rough sets techniquesAuthor(s): Gwanggil Jeon, Marco Anisetti, Ernesto Damiani, Olivier MongaPages: 1381-13842. Multiple dictionary pairs learning and sparse representation-based infrared image super-resolution with improved fuzzy clusteringAuthor(s): Xiaomin Yang, Wei Wu, Kai Liu, Weilong Chen, Zhili ZhouPages: 1385-13983. Retinex-based image enhancement framework by using region covariance filterAuthor(s): Fuyu Tao, Xiaomin Yang, Wei Wu, Kai Liu, Zhili Zhou, Yiguang LiuPages: 1399-14204. Early ramp warning using vehicle behavior analysisAuthor(s): Hua Cui, Zefa Wei, Xuan Wang, Xinxin Song, Pannong Li, Huansheng SongPages: 1421-14325. Vehicle trajectory clustering based on 3D information via a coarse-to-fine strategyAuthor(s): Huansheng Song, Xuan Wang, Cui Hua, Weixing Wang, Qi Guan, Zhaoyang ZhangPages: 1433-14446. Adaptive switching filter for impulse noise removal in digital contentAuthor(s): Jee Yon Lee, Sun Young Jung, Pyoung Won KimPages: 1445-14557. Sensor-based risk perception ability network design for drivers in snow and ice environmental freeway: a deep learning and rough sets approachAuthor(s): Wei Zhao, Liangjie Xu, Jing Bai, Menglu Ji, Troy RungePages: 1457-14668. Medical image denoising based on sparse dictionary learning and cluster ensembleAuthor(s): Jing Bai, Shu Song, Ting Fan, Licheng JiaoPages: 1467-14739. Segmentation and classification of hyperspectral images using CHV pattern extraction gridAuthor(s): Gokulakrishnan Gopalan, Tholkappia Arasu GovindarajanPages: 1475-149010. Chameleon-like weather presenter costume composite format based on color fuzzy modelAuthor(s): Pyoung Won KimPages: 1491-150011. Automatic vessel segmentation on fundus images using vessel filtering and fuzzy entropyAuthor(s): Huiqian Wang, Yuhao Jiang, Xiaoming Jiang, Jun Wu, Xiaomin YangPages: 1501-150912. A memetic algorithm based on MOEA/D for the examination timetabling problemAuthor(s): Yu Lei, Jiao Shi, Zhen YanPages: 1511-152313. On GPU–CUDA as preprocessing of fuzzy-rough data reduction by means of singular value decompositionAuthor(s): Salvatore Cuomo, Ardelio Galletti, Livia Marcellino, Guglielmo NavarraPages: 1525-153214. Real-time video processing for traffic control in smart city using Hadoop ecosystem with GPUsAuthor(s): M. Mazhar Rathore, Hojae Son, Awais Ahmad, Anand PaulPages: 1533-154415. Real-time implementation of a robust active control algorithm for narrowband signals suppressionAuthor(s): Jeakwan Kim, Minho Lee, Young-Sup LeePages: 1545-155416. An adaptive hybrid fuzzy-wavelet approach for image steganography using bit reduction and pixel adjustmentAuthor(s): Imran Shafi, Muhammad Noman, Moneeb Gohar, Awais Ahmad, Murad KhanPages: 1555-156717. Outgoing call recommendation using neural networkAuthor(s): Seokhoon KangPages: 1569-157618. Melanocytic and nevus lesion detection from diseased dermoscopic images using fuzzy and wavelet techniquesAuthor(s): Uzma Jamil, Shehzad Khalid, M. Usman Akram, Awais Ahmad, Sohail JabbarPages: 1577-159319. Wavelet-content-adaptive BP neural network-based deinterlacing algorithmAuthor(s): Jin Wang, Jechang JeongPages: 1595-160120. Stratified modeling in soft fuzzy topological structuresAuthor(s): S. E. Abbas, E. El-sanowsy, A. AtefPages: 1603-161321. On the power sequence of a fuzzy interval matrix with max-min operationAuthor(s): Yan-Kuen Wu, Chia-Cheng Liu, Yung-Yih LurPages: 1615-162222. A novel anomaly detection algorithm for sensor data under uncertaintyAuthor(s): Raihan Ul Islam, Mohammad Shahadat Hossain, Karl AnderssonPages: 1623-163923. Fuzzy multi-criteria decision making on combining fuzzy analytic hierarchy process with representative utility functions under fuzzy environmentAuthor(s): Yu-Jie WangPages: 1641-165024. BERA: a biogeography-based energy saving routing architecture for wireless sensor networksAuthor(s): Praveen Lalwani, Haider Banka, Chiranjeev KumarPages: 1651-1667y25. A parallel cooperative hybrid method based on ant colony optimization and 3-Opt algorithm for solving traveling salesman problemAuthor(s): Şaban Gülcü, Mostafa Mahi, Ömer Kaan Baykan, Halife KodazPages: 1669-168526. A novel versatile architecture for autonomous underwater vehicle’s motion planning and task assignmentAuthor(s): Somaiyeh Mahmoud Zadeh, David M. W. Powers, Karl SammutPages: 1687-171027. Forests of unstable hierarchical clusters for pattern classificationAuthor(s): Kyaw Kyaw HtikePages: 1711-171828. Fuzzy extensions of the DBScan clustering algorithmAuthor(s): Dino Ienco, Gloria BordognaPages: 1719-1730 Read more »
  • Soft Computing, Volume 22, Issue 4, February 2018
    Special issue on Quantitative Models and Weighted Automata1. Preface: Dedicated to the memory of Zoltán Ésik (1951–2016)Author(s): Manfred Droste, Kim G. LarsenPages: 10332. Characterizations of recognizable weighted tree languages by logic and bimorphismsAuthor(s): Zoltán Fülöp, Heiko VoglerPages: 1035-10463. A unifying survey on weighted logics and weighted automataAuthor(s): Paul Gastin, Benjamin MonmegePages: 1047-10654. Weighted restarting automataAuthor(s): Friedrich Otto, Qichao WangPages: 1067-10835. On decidability of recursive weighted logicsAuthor(s): Kim G. Larsen, Radu Mardare, Bingtian XuePages: 1085-11026. A generalized partition refinement algorithm, instantiated to language equivalence checking for weighted automataAuthor(s): Barbara König, Sebastian KüpperPages: 1103-11207. Weighted finite automata with outputAuthor(s): Jelena Ignjatović, Miroslav Ćirić, Zorana JančićPages: 1121-11388. Compositionality for quantitative specificationsAuthor(s): Uli Fahrenberg, Jan Křetínský, Axel Legay, Louis-Marie TraonouezPages: 1139-11589. Objective reduction for many-objective optimization problems using objective subspace extractionAuthor(s): Naili Luo, Xia Li, Qiuzhen LinPages: 1159-117310. Efficient and secure outsourced approximate pattern matching protocolAuthor(s): Xiaochao Wei, Minghao Zhao, Qiuliang XuPages: 1175-118711. Commutative deductive systems of pseudo-BCK-algebrasAuthor(s): Lavinia Corina CiunguPages: 1189-120112. Reduced axioms for the propositional logics induced by basic algebrasAuthor(s): Ivan Chajda, Miroslav KolaříkPages: 1203-120713. The importance of implementation details and parameter settings in black-box optimization: a case study on Gaussian estimation-of-distribution algorithms and circles-in-a-square packing problemsAuthor(s): Peter A. N. Bosman, Marcus GallagherPages: 1209-122314. Nearest neighbor search with locally weighted linear regression for heartbeat classificationAuthor(s): Juyoung Park, Md Zakirul Alam Bhuiyan, Mingon Kang, Junggab SonPages: 1225-123615. New distance measures on hesitant fuzzy sets based on the cardinality theory and their application in pattern recognitionAuthor(s): Fangwei Zhang, Shuyan Chen, Jianbo Li, Weiwei HuangPages: 1237-124516. Research on the measurement and evaluation of trusted cloud serviceAuthor(s): Zifei Ma, Rong Jiang, Ming Yang, Tong Li, Qiujin ZhangPages: 1247-126217. Self-adaptive differential evolution algorithm with hybrid mutation operator for parameters identification of PMSMAuthor(s): Chuan Wang, Yancheng Liu, Xiaoling Liang, Haohao Guo, Yang ChenPages: 1263-128518. Heterogeneous investment in spatial public goods game with mixed strategyAuthor(s): Hong Ding, Yao Zhang, Yizhi Ren, Benyun Shi, Kim-Kwang Raymond ChooPages: 1287-129419. A novel technical analysis-based method for stock market forecastingAuthor(s): Yuh-Jen Chen, Yuh-Min Chen, Shiang-Ting Tsao, Shu-Fan HsiehPages: 1295-131220. APDDE: self-adaptive parameter dynamics differential evolution algorithmAuthor(s): Hong-bo Wang, Xue-na Ren, Guo-qing Li, Xu-yan TuPages: 1313-133321. Adaptive harmony search with best-based search strategyAuthor(s): Zhaolu Guo, Huogen Yang, Shenwen Wang, Caiying Zhou, Xiaosheng LiuPages: 1335-134922. Face image retrieval: super-resolution based on sketch-photo transformationAuthor(s): Shu Zhan, Jingjing Zhao, Yucheng Tang, Zhenzhu XiePages: 1351-136023. Study on centroid type-reduction of general type-2 fuzzy logic systems with weighted enhanced Karnik–Mendel algorithmsAuthor(s): Yang Chen, Dazhi WangPages: 1361-1380 Read more »
  • Neural Networks, Volume 101, May 2018
    1. Neural network for nonsmooth pseudoconvex optimization with general convex constraintsAuthor(s): Wei Bian, Litao Ma, Sitian Qin, Xiaoping XuePages: 1-142. Neural electrical activity and neural network growthAuthor(s): F.M. GafarovPages: 15-243. Unified synchronization criteria in an array of coupled neural networks with hybrid impulsesAuthor(s): Nan Wang, Xuechen Li, Jianquan Lu, Fuad E. AlsaadiPages: 25-324. Delay-dependent dynamical analysis of complex-valued memristive neural networks: Continuous-time and discrete-time casesAuthor(s): Jinling Wang, Haijun Jiang, Tianlong Ma, Cheng HuPages: 33-465. Salient object detection based on multi-scale contrastAuthor(s): Hai Wang, Lei Dai, Yingfeng Cai, Xiaoqiang Sun, Long ChenPages: 47-56     6. Manifold regularized matrix completion for multi-label learning with ADMMAuthor(s): Bin Liu, Yingming Li, Zenglin XuPages: 57-677. Effective neural network training with adaptive learning rate based on training lossAuthor(s): Tomoumi Takase, Satoshi Oyama, Masahito KuriharaPages: 68-788. Robust Latent Regression with discriminative regularization by leveraging auxiliary knowledgeAuthor(s): Jianwen Tao, Di Zhou, Bin ZhuPages: 79-939. Distributed support vector machine in master–slave modeAuthor(s): Qingguo Chen, Feilong CaoPages: 94-10010. Designing a stable feedback control system for blind image deconvolutionAuthor(s): Shichao Cheng, Risheng Liu, Xin Fan, Zhongxuan LuoPages: 101-112 Read more »
  • IEEE Transactions on Cognitive and Developmental Systems, Vol. 10, No. 1, March 2018
    1. Affordances in Psychology, Neuroscience, and Robotics: A SurveyAuthor(s): L. Jamone, E. Ugur, A. Cangelosi, L. Fadiga, A. Bernardino, J. Piater and J. Santos-VictorPages: 4-252. A Logic-Based Computational Framework for Inferring Cognitive AffordancesAuthor(s): V. Sarathy and M. Scheutz,Pages: 26-433. Toward Lifelong Affordance Learning Using a Distributed Markov ModelAuthor(s): A. J. Glover and G. F. WyethPages: 44-554. Bootstrapping Relational Affordances of Object Pairs Using TransferAuthor(s): S. Fichtl, D. Kraft, N. Krüger and F. GuerinPages: 56-715. A Modular Dynamic Sensorimotor Model for Affordances Learning, Sequences Planning, and Tool-UseAuthor(s): R. Braud, A. Pitti and P. GaussierPages: 72-876. Artificial Life Environment Modeled by Dynamic Fuzzy Cognitive MapsAuthor(s): L. V. R. Arruda, M. Mendonca, F. Neves, I. R. Chrun and E. I. PapageorgiouPages: 88-1017. Bootstrapping Q-Learning for Robotics From Neuro-Evolution ResultsAuthor(s): M. Zimmer and S. DoncieuxPages: 102-119 Read more »
WordPress RSS Feed Retriever by Theme Mason

AI ML MarketPlace

  • AI fake face website launched
    A software developer has created a website that generates fake faces, using artificial intelligence (AI). generates a new lifelike image each time the page is refreshed, using technology developed by chipmaker Nvidia. Some visitors to the website say they have been amazed by the convincing nature of some of the fakes, although others are more clearly artificial. And many of them have gone on to post some of the fake faces on social media. Nvidia developed a pair of adversarial AI programs to create and then critique the images, in 2017. The company later made these programs open source, meaning they are publicly accessible. Image copyrightTHISPERSONDOESNOTEXIST.COM Image caption Not all faces on the website are convincingly human Realistic fakes As the quality of synthetic speech, text and imagery improves, researchers are encountering ethical dilemmas about whether to share their work.     Media caption Why these faces do not belong to 'real' people Last week, the Elon Musk backed OpenAI research group announced it had created an artificially intelligent "writer". But the San Francisco group took the unusual step of not releasing the technology behind the project publicly. "It's clear that the ability to generate synthetic text that is conditioned on specific subjects has the potential for significant abuse," the group said in a statement to AI blog Synced. Read Source Article:BBC News In Collaboration with HuntertechGlobal Read more »
  • Causal disentanglement is the next frontier in AI
    Recreating the human mind's ability to infer patterns and relationships from complex events could lead to a universal model of artificial intelligence.   A major challenge for artificial intelligence (AI) is having the ability to see past superficial phenomena to guess at the underlying causal processes. New research by KAUST and an international team of leading specialists has yielded a novel approach that moves beyond superficial pattern detection. Humans have an extraordinarily refined sense of intuition or inference that give us the insight, for example, to understand that a purple apple could be a red apple illuminated with blue light. This sense is so highly developed in humans that we are also inclined to see patterns and relationships where none exist, giving rise to our propensity for superstition. This type of insight is such a challenge to codify in AI that researchers are still working out where to start: yet it represents one of the most fundamental difference between natural and machine thought. Five years ago, a collaboration between KAUST-affiliated researchers Hector Zenil and Jesper Tegnér, along with Narsis Kiani and Allan Zea from Sweden's Karolinska Institutet, began adapting algorithmic information theory to network and systems biology in order to address fundamental problems in genomics and molecular circuits. That collaboration led to the development of an algorithmic approach to inferring causal processes that could form the basis of a universal model of AI. "Machine learning and AI are becoming ubiquitous in industry, science and society," says KAUST professor Tegnér. "Despite recent progress, we are still far from achieving general purpose machine intelligence with the capacity for reasoning and learning across different tasks. Part of the challenge is to move beyond superficial pattern detection toward techniques enabling the discovery of the underlying causal mechanisms producing the patterns." This causal disentanglement, however, becomes very challenging when several different processes are intertwined, as is often the case in molecular and genomic data. "Our work identifies the parts of the data that are causally related, taking out the spurious correlations and then identifies the different causal mechanisms involved in producing the observed data," says Tegnér. The method is based on a well-defined mathematical concept of algorithmic information probability as the basis for an optimal inference machine. The main difference from previous approaches, however, is the switch from an observer-centric view of the problem to an objective analysis of the phenomena based on deviations from randomness. "We use algorithmic complexity to isolate several interacting programs, and then search for the set of programs that could generate the observations," says Tegnér. The team demonstrated their method by applying it to the interacting outputs of multiple computer codes. The algorithm finds the shortest combination of programs that could construct the convoluted output string of 1s and 0s. "This technique can equip current machine learning methods with advanced complementary abilities to better deal with abstraction, inference and concepts, such as cause and effect, that other methods, including deep learning, cannot currently handle," says Zenil. Read more at: Source: In collaboration with HuntertechGlobal Read more »
  • Trump wants better AI. He also wants less immigration. He can’t have both.
    President Donald Trump released a splashy new plan for American artificial intelligence last week. High on enthusiasm, low on details, its goal is to ramp up the rate of progress in AI research so the United States won’t get outpaced by countries like China. Experts had been warning for months that under Trump, the US hasn’t been doing enough to maintain its competitive edge. Now, it seems, Trump has finally got the memo. His executive order, signed February 11, promises to “drive technological breakthroughs ... in order to promote scientific discovery, economic competitiveness, and national security.” Sounds nice, but unfortunately, there’s a problem. America’s ability to achieve that goal is predicated on its ability to attract and retain top talent in AI, much of which comes from outside the US. There’s a clear tension between that priority and another one of Trump’s objectives: cutting down on immigration, of both the legal and illegal varieties. Trump has spent the past two years of his presidency pushing away foreign-born scientists by means of restrictive visa policies. (Yes, he ad-libbed during the State of the Union that he might want more legal immigrants, but it’s really not clear how serious he was.) He’s also alienated them through his rhetoric — his decision to declare a national emergency to build a border wall is just the latest example. The result is a brain drain that academic research labs and tech companies alike have bemoaned. If the Trump administration really wants to reverse this trend and win the global AI race, it’s going to have to relax its anti-immigrant posture. The visa system would be a good place to start. Writing in Wired last week, Oren Etzioni, CEO of the Allen Institute for Artificial Intelligence, called on Trump “to include a special visa program for AI students and experts.” Etzioni argued that “providing 10,000 new visas for AI specialists, and more for experts in other STEM fields, would revitalize our country’s research ecosystem, empower our country’s innovation economy, and ensure that the United States remains a world superpower in the coming decades.” As Etzioni noted, the Trump administration has so far made it harder for foreigners to get H-1B visas, which allow highly skilled workers to perform specialized jobs. The visa process takes longer than it used to, lawyers are reporting that more applications are getting denied, and computer programmers no longer qualify as filling a specialty occupation under the H-1B program. Even those lucky ones who do get the coveted visas may have a hard time putting down roots in the US, because the administration has signaled it may nix the H-4EAD program, which allows the spouses of H-1B holders to live and work in the country. It’s hard to overstate how anxiety-provoking all this can be for authorized immigrant families, whether or not they work in AI. Imagine not knowing, for months at a stretch, whether you’ll get to keep working in the US, or whether you and your partner will be able to live in the same country. Facing this kind of uncertainty, some would-be applicants prefer to pursue less stressful options — in countries that actually seem to want them. Despite getting bogged down in the courts, Trump’s travel ban also impacted hundreds of scientists from Muslim-majority countries — and deprived the US of their contributions. “Science departments at American universities can no longer recruit Ph.D. students from Iran — a country that … has long been the source of some of our best talent,” MIT professor Scott Aaronson wrote on his blog in 2017. “If Canada and Australia have any brains,” he added, “they’ll snatch these students, and make the loss America’s.” Canada has done just that. The country, which hosts some of the very best AI researchers in tech hubs like Toronto and Montreal, was the first in the world to announce a national AI plan: In March 2017, it invested $125 million in the Pan-Canadian AI Strategy. That summer, it also rolled out a fast-track visa option for highly skilled foreign workers. And as Politico reported, in a survey conducted by tech incubator MaRS, “62 percent of Canadian companies polled said they had noticed an increase in American job applications, particularly after the election.” Canadian universities also reported a major uptick in interest from foreign students. University of Toronto professors, for example, reported seeing a 70 percent increase in international student applications in fall 2017, compared to 2016. Taken together, Trump’s immigration policies have hurt the US when it comes to STEM in general and AI in particular. But his executive order gives no sign that the administration understands that. It features the words “immigration” and “visa” exactly zero times. If anything, it seems to double down on Trump’s “America first” philosophy. A section titled “AI and the American Workforce” says heads of agencies that provide educational grants must prioritize AI programs and that those programs must give preference to American citizens. This approach fits with Trump’s overall belief that immigrants, both legal and illegal, take jobs from US workers. He’s just applying that principle to the field of AI researchers. Yet the primary stated goal of Trump’s AI strategy is not to protect the career prospects of individual American workers, but to protect Americans at large from being overtaken by other countries in the AI race. Of course, those two projects may converge to some extent. But there will also be instances where they diverge — in the context of hiring decisions, say. In those instances, it’s most effective to choose the candidate (wherever she comes from) who’ll do the best job at beefing up America’s AI strategy, so that it can ultimately benefit a vast number of Americans. Maintaining the US advantage in AI has become an increasingly urgent project since last year, when China declared its intention to become the world leader in the field by 2030. Although Trump’s executive order avoids mentioning China by name, a US Defense Department document on AI strategy released the very next day was not so circumspect. As the public summary of a strategy that was developed last year, it offers a window into the motivations that are driving the Trump administration now. And it casts the situation in pretty dire terms: Other nations, particularly China and Russia, are making significant investments in AI for military purposes, including in applications that raise questions regarding international norms and human rights. These investments threaten to erode our technological and operational advantages and destabilize the free and open international order. The United States, together with its allies and partners, must adopt AI to maintain its strategic position, prevail on future battlefields, and safeguard this order. There’s good reason to be concerned about China winning a global AI race. Already, it’s using AI surveillance technologies to become one of the most repressive countries on the planet. For the US to let in fewer foreign researchers would be to risk more of them being employed by China, which could mean its military makes key advances first. Americans who tend to worry deeply about the risks AI poses to humanity may argue that slowing AI growth means we forestall those risks. But slowing growth in the US won’t impede growth worldwide; it just means it’ll happen elsewhere. In fact, there’s a compelling argument to be made that more immigrants working in American AI would allow firms that are concerned with AI safety to develop safer methods faster. That’s not just because more people power will yield more research and development, but because immigrants might be more attuned to racial, ethnic, and class disparities in the way AI risk gets distributed. Those disparities have been too often glossed over in Silicon Valley. At least one firm aiming to build safe artificial general intelligence, OpenAI, seems cognizant of that. “Our goal is to build safe AGI which benefits the world, which means we want our team to be maximally representative of the world,” its website states, before specifically noting: “We sponsor visas including cap-exempt H-1Bs.” This diversity-embracing approach is a far cry from the one laid out in the president’s executive order. Ultimately, Trump can ensure America’s place as an AI superpower. Or he can try to keep AI jobs out of the hands of non-Americans. But he has to choose. Read Source Article:Vox In Collaboration with HuntertechGlobal   Read more »
  • Robot Journo: China's Xinhua to Unveil First Woman AI News Presenter
    China's Xinhua News Agency and the search engine Shougou Company who together collaborated in the development of the artificial intelligence anchor, called Xin Xiaomeng. Adding another feather to its tech savvy cap, China on Tuesday unveiled the world's first female AI news presenter. The announcement was made by China's Xinhua News Agency and the search engine Shougou Company who together collaborated in the development of the artificial intelligence anchor, called Xin Xiaomeng.  Xiaomeng will debut in March during the upcoming Two Sessions meetings, the annual Parliamentary meetings held in China. View image on Twitter   As remarkable as the AI presenter is, this is not the first robot news presenter in the world. In 2018, China became the first country in the world to develop the first of its kind AI 'journalist', a male news presenter by the name of Qui Hao. The specimen was unveiled during China’s annual World Internet Conference in November. But China's experiments with AI reporters and journalists has been going on for some years. In 2012, the University of Science and Technology of China started developing a woman robot by the name of 'Jia Jia'. The robot was unveiled in 2017 when she took questions from AI expert and Wired co-founder Kevin Kelly. The interaction was filmed and released by Xinhua. The awkward conversation and the ineptitude of the robot in answering questions with speed and clarity left many wondering if robots could indeed compete with humans, especially in subjective-perceptive fields like journalism that relies heavily on human instincts and discretion of journalists.However, Xinhua stated that ever since launching AI employees in November 2018, the robots have filed 3,400 reports. In fact, China had even introduced an AI 'intern' reporter in 2017 during that year's Two Sessions meetings. The robot was called 'Inspire'. With rapid developments in the field of AI, China is soon emerging as a world leader in the sector, edging out countries like the US and Japan. The Boston Consulting Group’s study Mind the (AI) Gap: Leadership Makes the Difference, published in December last year, found that 85 percent of Chinese companies are active players in the AI sector. The New Generation Artificial Intelligence Development Plan introduced in 2017 is also responsible for the rapid growth of AI in China. With its large array of tech-startups, US is currently at the second place in terms of AI expansion as a sector, after China. Even private players such as Google have started taking an interest. In 2017, Google financed the development of 'RADAR' (Reporters And Data And Robots), a software that will gather, automate and produce news reports. It was developed by British news agency Press Association at a cost of $805,000.South Korea’s Yonhap News Agency also introduced an automated system of reporting called 'Soccerbot' which will dedicatedly produce football related news. With larger and more concerted efforts being made to develop AI through the world, robot news presenters could soon become a common reality, much like Siri or Alexa. But is the job sector, especially in countries like India, ready to accommodate these new robotic players?   Read Source Article: News18 In Collaboration with HuntertechGlobal   Read more »
  • The AI That Can Write A Fake News Story From A Handful Of Words
    The potential for the OpenAI software to be able to be able to near-instantly create fake news articles comes during global concerns over technology's role in the spread of disinformation. OpenAI, an artificial intelligence research group co-founded by billionaire Elon Musk, has demonstrated a piece of software that can produce authentic-looking fake news articles after being given just a few pieces of information. In an example published Thursday by OpenAI, the system was given some sample text: "A train carriage containing controlled nuclear materials was stolen in Cincinnati today. Its whereabout are unknown." From this, the software was able to generate a convincing seven-paragraph news story, including quotes from government officials, with the only caveat being that it was entirely untrue. "The texts that they are able to generate from prompts are fairly stunning," said Sam Bowman, a computer scientist at New York University who specializes in natural language processing and who was not involved in the OpenAI project, but was briefed on it. "It's able to do things that are qualitatively much more sophisticated than anything we've seen before." OpenAI is aware of the concerns around fake news, said Jack Clark, the organization's policy director. "One of the not so good purposes would be disinformation because it can produce things that sound coherent but which are not accurate," he said. As a precaution, OpenAI decided not to publish or release the most sophisticated versions of its software. It has, however, created a tool that lets policymakers, journalists, writers and artists experiment with the algorithm to see what kind of text it can generate and what other sorts of tasks it can perform. The potential for software to be able to be able to near-instantly create fake news articles comes during global concerns over technology's role in the spread of disinformation. European regulators have threatened action if tech firms don't do more to prevent their products helping sway voters, and Facebook has been working since the 2016 U.S. election to try and contain disinformation on its platform.   Clark and Bowman both said that, for now, the system's abilities are not consistent enough to pose an immediate threat. "This is not a shovel-ready technology today, and that's a good thing," Clark said. Unveiled in a paper and a blog post Thursday, OpenAI's creation is trained for a task known as language modeling, which involves predicting the next word of a piece of text based on knowledge of all previous words, similar to how auto-complete works when typing an email on a mobile phone. It can also be used for translation, and open-ended question answering. One potential use is helping creative writers generate ideas or dialog, Jeff Wu, a researcher at OpenAI who worked on the project, said. Others include checking for grammatical errors in texts, or hunting for bugs in software code. The system could be fine-tuned to summarize text for corporate or government decision makers further in the future, he said. In the past year, researchers have made a number of sudden leaps in language processing. In November, Alphabet Inc.'s Google unveiled a similarly multi-talented algorithm called BERT that can understand and answer questions. Earlier, the Allen Institute for Artificial Intelligence, a research lab in Seattle, achieved landmark results in natural language processing with an algorithm called Elmo. Bowman said BERT and Elmo were "the most impactful development" in the field in the past five years. By contrast, he said OpenAI's new algorithm was "significant" but not as revolutionary as BERT. COMMENT Although co-founded by Musk, he stepped down from OpenAI's board last year. He'd helped kickstart the non-profit research organization in 2016 along with Sam Altman and Jessica Livingston, the Silicon Valley entrepreneurs behind startup incubator Y Combinator. Other early backers of OpenAI include Peter Thiel and Reid Hoffman. Read source Article NDTV In Collaboration with HuntertechGlobal Read more »
WordPress RSS Feed Retriever by Theme Mason

Artificial Intelligence Weekly News

  • Artificial Intelligence Weekly - Artificial Intelligence News #98 - Feb 21st 2019
    In the News The Rise of the Robot Reporter As reporters and editors find themselves the victims of layoffs at digital publishers and traditional newspaper chains alike, journalism generated by machine is on the rise. Getting smart about the future of AI Artificial intelligence is a primary driver of possibilities and promise as the Fourth Industrial Revolution unfolds. Sponsor Add Audible AI to Any Web Page in Just 5 Lines of HTML Neon adds Audible AI to all your pages quickly and easily! Empower your website users to gather helpful information by using voice commands or by typing. Equip your site so users can ask for real-time Q&A, conversions, math solutions, language translation, transcription & more! Customizable! Watch our Audible AI demo to learn how. Learning Better Language Models and Their Implications OpenAI trained GPT2, a language generation model that achieved surprisingly good results (see article for examples). Seeing this performance, OpenAI decided not to open-source their best model for fear it might be mis-used (online trolling, fake news, cyber bullying, spam...) List of Machine Learning / Deep Learning conferences in 2019 Perspectives on issues in AI Governance Report by Google focusing on 5 areas for clarification: explainability, fairness, safety, human-AI collaboration and liability Software tools & code Introducing PlaNet Instead of using traditional RL approaches, Google has trained an agent to "learn a world model" and thus become more efficient at planning ahead. Troubleshooting Deep Neural Networks A field guide to fixing your model Hardware Edge TPU Devices The Edge TPU is a small ASIC designed by Google that perform ML inferencing on low-power devices. For example, it can execute MobileNet V2 at 100+ fps in a power efficient manner. Facebook is Working on Its Own Custom AI Silicon Workplace Succeeding as a data scientist in small companies/startups Some thoughts Will AI achieve consciousness? Wrong question When Norbert Wiener, the father of cybernetics, wrote his book The Human Use of Human Beings in 1950, vacuum tubes were still the primary electronic building blocks, and there were only a few actual computers in operation. About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #97 - Feb 7th 2019
    In the News DeepMind wants to teach AI to play a card game that’s harder than Go Hanabi is a card game that relies on theory of mind and a higher level of reasoning than either Go or chess—no wonder DeepMind’s researchers want to tackle it next. In the news this week... China is said to be worried an AI arms race could lead to accidental war. More Google says it wants rules for the use of AI—kinda, sorta. More Is China’s corruption-busting AI system ‘Zero Trust’ being turned off for being too efficient? More Sponsor Build Better Bots with Our Neon Conversational AI SDK Finally, a white label solution for your polylingual conversational AI needs. Neon provides advanced Natural Language Understanding so you can build custom audio-responsive devices, AI personal assistants, home automation, corporate apps and more…with real-time translated responses! Watch our AI in action. Learning Cameras that understand: portrait mode and Google Lens On-device ML and computer vision advances will help make camera sensors a lot smarter. Today it's all about "computational photography", but tomorrow cameras will be able to anticipate and understand our needs and context. An AI is playing Pictionary to figure out how the world works Forget Go or StarCraft—guessing the phrase behind a drawing will require machines to gain some understanding of the way concepts fit together in the real world. Software tools & code Papers with code Great new resource that provides summaries and links to Machine Learning papers along with the corresponding code and evaluation tables. Multi-label Text Classification using BERT Diversity in Faces IBM Research releases ‘Diversity in Faces’ dataset to advance study of fairness in Facial Recognition systems Running TensorFlow at Petascale and Beyond Uber AresDB Introducing Uber’s GPU-powered Open Source, Real-time Analytics Engine Workplace Data Scientist Salaries and Jobs in Europe Here is what a recent report says about job opportunities for Data Scientists across Europe including salaries and benefits, job motivations, programming languages used, tech skills and what people want most from their work. Your AI skills are worth less than you think Some thoughts 10 TED Talks on AI and machine learning How will AI reshape your career? Your health? Your ability to tell real from fake video? Recent TED talks explore some fascinating AI questions About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #96 - Jan 31st 2019
    In the News Deepmind's AlphaStar beats pro Starcraft player Starcraft has been a focus for AI due to its higher complexity vs chess or go: imperfect information, real-time, a lot more things happening... Granted, Alphastar was maybe not on a level-playing fields with humans. But this is still an impressive feat. Virtual creators aren’t AI — but AI is coming for them Lil Miquela, the A.I. generated Instagram superstar, may just be the beginning. Amazon's delivery robot Scout Amazon has revealed Scout, a six-wheeled knee-height robot designed to autonomously deliver products to Amazon customers. Sponsor Quickly Enable Your Solutions with Conversational AI Tech Don’t get left in the AI dust. You need fast, sophisticated AI enablement—and we’ve got it. From real-time transcription and language translation to smart alerts and database integration, our patented technologies can put your apps and devices ahead of the pack. Watch our demos to learn how. Learning We analyzed 16,625 papers to figure out where AI is headed next Our study of 25 years of artificial-intelligence research suggests the era of deep is coming to an end. Practical Deep Learning for coders 2019 This is the 3rd iteration of's great online learning resources for Coders. There are seven lessons, each around 2 hours long, covering Computer vision (classification, localization, key-points), NLP (language modeling, document classification) and tabular data (both categorical and continuous). Why are Machine Learning projects so hard to manage? "I’ve watched lots of companies attempt to deploy machine learning — some succeed wildly and some fail spectacularly. One constant is that machine learning teams have a hard time setting goals and setting expectations. Why is this?" Software tools & code Natural Questions — by Google Google released a dataset containing around 300,000 questions along with human-annotated answers from Wikipedia pages. Useful for Question Answering Research. The state of the octoverse: machine learning Github's review of the most popular languages, frameworks and tools for Machine Learning. Transformer-XL: Unleashing the Potential of Attention Models Introducing Transformer-XL, a novel architecture that enables natural language understanding beyond a fixed-length context... Hardware This robot can probably beat you at Jenga—thanks to its understanding of the world Industrial machines could be trained to be less clumsy if we gave them a sense of touch and a better sense of real-world physics. Some thoughts The AI threat to open societies — by George Soros In an age of populist nationalism, open societies have increasingly come under strain. But the threat of atavistic ideological movements pales in comparison to that posed by powerful new technologies in the hands of authoritarians. A.I. could worsen health disparities In a health system riddled with inequity, we risk making dangerous biases automated and invisible. About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #95 - Jan 24th 2019
    In the News AI is sending people to jail—and getting it wrong Using historical data to train risk assessment tools could mean that machines are copying the mistakes of the past. Three charts show how China’s AI industry is propped up by three companies More than half of the country’s major AI players have funding ties that lead back to Baidu, Alibaba, and Tencent. Sponsor Is There Bias in Your AI Model? Do You Know How It Got There? Bias comes in a variety of forms, all of them potentially damaging to the efficacy of your ML algorithm. Our Chief Data Scientist specializes in training data bias, the source of most headlines about AI failures. Our overview – Four Types of AI Bias – is a guide for detecting and mitigating bias. Learning Few-shot learning Thoughts on progress made and challenges ahead in few-shot learning (i.e. learning from tiny datasets). Slides by Hugo Larochelle (Google Brain) What can neural networks learn? "Neural networks are famously difficult to interpret. It’s hard to know what they are actually learning when we train them. Let’s take a closer look and see whether we can build a good picture of what’s going on inside." Looking Back at Google’s AI Research Efforts in 2018 Software tools & code Machine Learning for Kids Web-based list of ML projects aimed at children aged 8 to 16. They cover simple computer vision, NLP, game mechanics, and are available on "scratch", a coding platform for children. Uber Manifold Model-Agnostic Visual Debugging Tool for Machine Learning What’s coming in TensorFlow 2.0 Workplace Demand and salaries for Data Scientists continue to climb Data-science job openings are expanding faster than the number of technologists looking for them Airbnb Data Science Interview Questions Some thoughts How AI will turn us all into Filmmakers About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #94 - Jan 10th 2019
    In the News Cheaper AI for everyone is the promise with Intel and Facebook’s new chip Companies hoping to use artificial intelligence should benefit from more efficient chip designs. Finland’s grand AI experiment Inside Finland’s plan to train its whole population in artificial intelligence. World's largest AI startup readies $2B fundraising Sponsor The first AI/machine learning course with job guarantee Work with the latest AI applications after completing Springboard's self-paced, online machine learning course. Weekly personal calls with your own AI/machine learning expert. Personalized career coaching. Build a portfolio of meaningful projects that will get you hired. Get a job or your tuition back with the proven Springboard job guarantee. Learning High-performance medicine On the convergence of human and artificial intelligence Does AI make strong tech companies stronger? A.I. needs lots of data to work well, which leads to virtuous circles (more data => better AI => better product => more data) that can benefit large established tech companies. Is this true though? Unprovability comes to machine learning Scenarios have been discovered in which it is impossible to prove whether or not a machine-learning algorithm could solve a particular problem. This finding might have implications for both established and future learning algorithms. Lessons Learned at Instagram Stories and Feed Machine Learning Instagram's recommender system serves over 1 billion users on a regular basis for feed and stories ranking as well as post recommendations and smart prefetching. Here are a few lessons learnt along the way of building this ML pipeline. Software tools & code Designing an audio adblocker for radio and podcasts Adblock Radio detects audio ads with machine-learning and Shazam-like techniques. The core engine is open source: use it in your radio product! You are welcome to join efforts to support more radios and podcasts. What Kagglers are using for Text Classification About TextCNN, Bidirectional RNNs and Attention Models. 10 Data Science tools I explored in 2018 Tensorflow Privacy Library for training machine learning models with privacy for training data. This makes use of differential privacy. Workplace Does my startup data team need a Data Engineer? The role of the data engineer in a startup data team is changing rapidly. Are you thinking about it the right way? Some thoughts How big data has created a big crisis in science About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #93 - Dec 27th 2018
    In the News Should we be worried about computerized Facial Recognition? The technology could revolutionize policing, medicine, even agriculture—but its applications can easily be weaponized. AlphaZero: Shedding new light on the grand games of chess, shogi and Go Introducing the full evaluation of AlphaZero on how it learns each game to become the strongest player in history for each, despite starting its training from random play, with no in-built domain knowledge but the basic rules of the game. Learning 10 Exciting Ideas of 2018 in NLP Unsupervised MT, pre-trained language models, common sense inference datasets, meta-learning, robust unsupervised learning, understanding representations, clever auxiliary tasks, inductive bias and more How AI Training Scales "We’ve discovered that the gradient noise scale, a simple statistical metric, predicts the parallelizability of neural network training on a wide range of tasks. Since complex tasks tend to have noisier gradients, increasingly large batch sizes are likely to become useful in the future, removing one potential limit to further growth of AI systems. More broadly, these results show that neural network training need not be considered a mysterious art, but can be rigorized and systematized." Data Science vs Engineering: Tension Points Current state of collaboration around building and deploying models, tension points that potentially arise, as well as practical advice on how to address these tension points. Using object detection for complex image classification scenarios For situations where scenes don't contain one main object or a simple scene, object detection can be used to improve the performance of computer vision algorithms. Comes with examples from the retail industry. The limitations of deep learning Software tools & code Text as Data Learn how to collect and analyze social media data using topic models, text networks, and word2vec with this open source version of the Text as Data class from Duke's Data Science program. Wav2letter++, the fastest open source speech system, and flashlight Open-sourced by Facebook: a new fully convolutional approach to automatic speech recognition and wav2letter++, the fastest state-of-the-art end-to-end speech recognition system available. Hardware A Full Hardware Guide to Deep Learning About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #92 - Dec 13th 2018
    In the News A radical new neural network design could overcome big challenges in AI Researchers borrowed equations from calculus to redesign the core machinery of deep learning so it can model continuous processes like changes in health. The friendship that made Google huge Coding together at the same computer, Jeff Dean and Sanjay Ghemawat changed the course of the company—and the Internet. Alibaba already has a voice assistant way better than Google’s It navigates interruptions and other tricky features of human conversation to field millions of requests a day. Also in the news... DeepMind has announced AlphaFold: an AI to predict the 3D structure of a protein based solely on its genetic sequence. More Waymo is introducing the Waymo One, a fully self-driving service in the Phoenix Metro area. More Learning Community-driven site listing recent and interesting papers to help you review the state-of-the-art in NLP, computer vision, game playing, program synthesis etc. AI Index 2018 Report Long and detailed report on A.I. research activity, industry activity and technical performance for 2018. Predicting the real-time availability of 200 million grocery items Details on Instacart's model to continuously monitor and predict the availability of 200 million grocery items. Software tools & code Deepdive into Facebook's open-source Reinforcement Learning platform "Facebook decided to open-source the platform that they created to solve end-to-end Reinforcement Learning problems at the scale they are working on. So of course I just had to try this 😉 Let’s go through this together on how they installed it and what you should do to get this working yourself." Machine Learning Basics - Gradient Boosting & XGBoost Like Random Forest, Gradient Boosting is another technique for performing supervised machine learning tasks, like classification and regression. Hardware Amazon’s homegrown chips threaten Silicon Valley giant Intel Amazon is upping its efforts to build its own CPUs, in particular what it calls the Inferentia, a chip specifically designed for ML inference. Workplace 3 common data science career transitions, and how to make them happen Some thoughts The deepest problem with deep learning Some reflections on an accidental Twitterstorm, the future of AI and deep learning, and what happens when you confuse a schoolbus with a snow plow. About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #91 - Nov 29th 2018
    In the News How cheap labor drives China’s A.I. ambitions Cheap manual labor is one of the current requirements for large AI developments. China has a lot of inexpensive labor force and is making it one of its key advantages to become the AI world leader by 2030. Is the Chinese billionaire Jack Ma using AI to create dystopian cities? "News that tech giant Ma is a member of Communist party of China should set alarm bells ringing – and not just in China" One of the fathers of AI is worried about its future Yoshua Bengio wants to stop talk of an AI arms race and make the technology more accessible to the developing world. Also in the news... US commerce department proposes new export restrictions on AI. Tweet, Report Fearful of gender bias, Google blocks gender based pronouns from Smart Compose. More Learning Easy-to-read summary of important AI research papers of 2018 Lack time to read research papers? Have a look at this great summary of some of the main ideas found in recent research papers, along with comments from the community. Beating the state-of-the-art in NLP with HMTL Multi-Task Learning is a general method in which a single architecture is trained towards learning several different tasks at the same time. Here's an example with such a model (HMTL) trained to beat the state-of-the-art on several NLP tasks. Why bigger isn’t always better with GANs and AI art Looking at GANs less from a performance standpoint and more from an artistic one. Automated testing in the modern data warehouse "If data issues have lasting consequences, why are we less sophisticated at testing than our software developing counterparts?" Measuring what makes readers subscribe to The New York Times How the NYT builds fast and reusable econometric models in-house Software tools & code BigGAN A new state of the art in Image synthesis Best Deals in Deep Learning Cloud Providers Comparing prices to train models on GPUs and TPUs with AWS, Google, Paperspace, and others. Hardware Basic Graphics Processing Unit (GPU) design concepts Graphics pipeline, vector processing along with the other graphics operations. Useful to have in mind when wading into GPU-powered ML training. Some thoughts Portrait of Li Fei-Fei Former Professor at Stanford University in Computer Vision, now Chief Scientist for Google Cloud AI — on her "quest to make AI better for humanity". About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #90 - Nov 15th 2018
    In the News A.I. hits the barrier of meaning Machine learning algorithms don’t yet understand things the way humans do — with sometimes disastrous consequences. Why China can do AI more quickly and effectively than the US Harvard wants to school Congress about AI A tech boot camp will teach US politicians and policymakers about the potential, and the risks, of artificial intelligence. Also in the news... Microsoft's president: we need to regulate facial recognition tech. But AI is still very very stupid. Habana Labs raises $75M for its AI chips, DeepMap raises $60M for its HD maps for self-driving cars and Engineer.AI raises $29.5M for its AI+Humans software building platform. Learning Scaling Machine Learning at Uber with Michelangelo Some thoughts on the evolution of ML at Uber over the last three years: why and how they developed their Michelangelo platform along with the way they have been scaling ML to support large numbers of experiments Slow Learning The Economist and others have recently described how online programs like can "equip the masses to use AI." One of the protagonists, Sara Hooker, feels uneasy about this and gives here a firsthand description of what it really takes to be successful in this field. Stephen Wolfram on AGI (video) 2h lecture by Stephen Wolfram on what he learned from building Wolfram Alpha. Can we rule out near-term AGI? (video) Talk by Greg Brockman, founder of OpenAI. Software tools & code Open Images Dataset V4 New dataset released by Google. Around 10 million images with bounding boxes, visual relationships, and image-level labels for 20,000 distinct concepts Generating custom photo-realistic faces using AI Controlled image synthesis and editing using a novel TL-GAN model Facebook Horizon Open Source Applied Reinforcement Learning Platform Facebook zero-shot learning Using text to accurately identify images Some thoughts Is artificial intelligence set to become art’s next medium? Who’s going to profit most from the A.I. wave? About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! Artificial Intelligence Weekly This RSS feed is published on You can also subscribe via email. Read more »
  • Artificial Intelligence Weekly - Artificial Intelligence News #89 - Oct 25th 2018
    In the News Montreal has reinvented itself as the world's AI startup powerhouse Affordable office space and a community feel have helped Canada’s second-largest city develop into a thriving AI hotspot Your next doctor’s appointment might be with an AI A new wave of chatbots are replacing physicians and providing frontline medical advice—but are they as good as the real thing? AI startups, tech giants are at the center of the Chinese government's plans Also in the news... MIT Plans College for Artificial Intelligence, Backed by $1 Billion. More Baidu challenges Google with AI that performs language translation in realtime. More Tesla deploys massive new Autopilot neural net in v9, impressive new capabilities, report says. More Huawei releases its AI Strategy and full-stack, all-scenario AI portfolio. More Learning A global ethics study aims to help AI solve the self-driving “trolley problem” Millions of people in 233 countries weighed in on whose lives self-driving cars should prioritize, revealing how much ethics diverge across cultures. 5 ways Google Pixel 3 camera pushes the boundaries of computational photography Google launched its Google Pixel 3 line-up earlier this month. The improvement over the previous generation seems to be in the phone's photo-taking capabilities. Most of this improvement is achieved through software and machine learning. Airbnb: Applying Deep Learning to Airbnb Search ThŒe application to search ranking is one of the biggest machine learning success stories at Airbnb. Much of the initial gains were driven by a gradient boosted decision tree model. Œe gains, however, plateaued over time. Œis paper discusses the work done in applying neural networks in an aŠempt to break out of that plateau. Google fairness in ML A new course to teach people fairness in machine learning. It explores how human biases affect data sets. Software tools & code Tensorflow 2.0 Video on the upcoming changes in Tensorflow 2.0 with a side-by-side comparison with Pytorch. An Introduction to GPU Programming in Julia How to build your own AlphaZero AI using Python and Keras Teach a machine to learn Connect4 strategy through self-play and deep learning Some thoughts Will compression be Machine Learning’s killer app? About This newsletter is a collection of AI news and resources curated by @dlissmyr. If you find it worthwhile, please forward to your friends and colleagues, or share on your favorite network! Share on Twitter · Share on Linkedin · Share on Google+ Suggestions or comments are more than welcome, just reply to this email. Thanks! This RSS feed is published on You can also subscribe via email. Read more »
WordPress RSS Feed Retriever by Theme Mason

Marketing Artificial Intelligence Institute

  • The Impact on Sales as AI Goes Mainstream
    true recently released its State of Artificial Intelligence for Sales and Marketing 2018 Report. Based on responses from 600 business professionals, the report dives into what leaders need to know about the coming AI revolution. We’ve highlighted the three most important findings for marketers below, but download the full report for more information on what AI trends have leaders most concerned, most excited, and more. Read more »
  • Marketing’s Challenge with the AI Black Box
    Marketers have an AI problem. Read more »
  • What Happens to Marketing When AI Can Write Like Humans?
    The simplest way to understand artificial intelligence is to think about it as a set of technologies and algorithms that are designed to make machines smart, to give them humanlike capabilities (e.g. vision, hearing, speech, writing, understanding, movement). Read more »
  • How Artificial Intelligence Impacts Every Marketer’s Email Campaigns
    Your email campaigns are a direct channel to the consumer without gatekeepers or intermediaries—or so you might think. So many brands rely on email campaigns to do business directly with consumers who sign up to hear from them. However, mailbox providers have a huge influence on how effective your email marketing is. Your list may be composed of your customers and leads. But those customers and leads are also a mailbox provider’s users. Read more »
  • AI-Powered Competitive Intelligence Tool Announces $6 Million in Funding and New Industry Report
    Crayon, a competitive intelligence (CI) company, just announced $6 million in Series A funding for its AI-powered competitive intelligence tool. Read more »
  • How AI Improves Content Strategy and Increase Sales
    Picture the following: Read more »
  • How to Make Your Website Amazon Alexa’s Top Choice
    At the Marketing AI Institute, we read dozens of articles on artificial intelligence every week to uncover the most valuable ones for our subscribers and we curate them for you here. We call it 3 Links in 3 Minutes. Enjoy! Read more »
  • Research Proves the Benefits of Getting Ahead of the AI Curve
    “Artificial intelligence” is at peak buzzword status. As marketers, we see it everywhere: Affixed to the claims of the latest marketing tech, alongside warnings that it will eradicate jobs like content production, and peppered into predictions of coming trends. Many marketers are used to being bombarded with hype about “the next big thing.” In this case, they either become numb to mention of AI because it’s usually not actionable or approachable, or they’re disillusioned because it’s just a catchy term for tools that aren’t really AI. Read more »
  • Over 90% of Sales Professionals Expect AI to Improve Performance in 2019
    At the Marketing AI Institute, we read dozens of articles on artificial intelligence every week to uncover the most valuable ones for our subscribers and we curate them for you here. We call it 3 Links in 3 Minutes. Enjoy! Read more »
  • 60+ Speakers Scheduled to Present at the Marketing Artificial Intelligence Conference (MAICON)
    The Marketing Artificial Intelligence Conference (MAICON) agenda is now live featuring 40+ sessions and 60+ speakers over three days, July 16 - 18, 2019 in Cleveland, Ohio. MAICON includes five interactive workshops and 37 general and breakout sessions—all developed to help marketers understand, pilot and successfully scale artificial intelligence. Presenters will explore the next frontier in digital marketing transformation, covering topics such as: advertising, analytics, content marketing, email marketing, ethics, robotics, sales, strategy, voice and more. Read more »
WordPress RSS Feed Retriever by Theme Mason